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BACKGROUND 


During  mid  1977  Lt.  General  Thomas  W.  Morgan.  Commander  of 
the  Air  Force  Space  and  Missiles  Organization,  SAMSO,  met  with 
key  executives  of  the  space  industry  to  discuss  fundamental  issues 
that  would  enhance  spacecraft  mission  success.  These  leaders  col- 
lectively identified  a wide  range  of  key  elements  that  were  critical 
to  successful  space  missions.  The  participants  and  a summary 
overview  of  the  Executive  Sessions  are  included  in  Appendix  B of 
these  proceedings.  Three  industry  associations  (NSIA,  AIA,  and 
ADPA)  accepted  the  challenge  to  explore  these  issues  in  greater 
detail  in  a conference/workshop  environment  to  examine  key 
elements  and  factors  critical  to  space  mission  success.  The  intent 
of  the  sessions  was  to  identify  tools,  techniques  and  methods  in 
acquisition  and  procurement  to  minimize  and  control  risk  through 
improved  management  and  practical  implemention  of  all  actions 
affecting  the  achievement  of  Mission  Assurance. 
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ABSTRACT 


These  proceedings  tell  only  a small  part  of  the  tremendous  effort  behind  the  BAMSO/Industry  Conference 
on  Mission  Assurance,  25-27  April  1978,  as  many  months  of  preparation,  study  and  research  were  required 
by  speakers,  co-chairmen  and  workshop  participants.  This  volume  contains  the  proceedings  of  a unique 
conference  that  looked  at  ways  to  enhance  mission  success  from  a "Systems  Approach"  rather  than  from  a 
specialized  or  parochial  standpoint.  The  formal  speeches  contained  herein  set  the  stage  for  the  panel  and 
workshop  sessions  by  providing  management  overview  and  policy  perspective.  The  main. points  of  the  panel 
and  workshop  sessions  are  summarized  below: 


Panels 


• The  piece  parts  panel  presented  an  overview  of  the  various  aspects  of  piece  parts  management  for 
Mission  Assurance  from  the  perspective  of  SAMSO,  NASA,  DESC,  and  Industry. 


• The  Design  Panel  discussed,  from  an  executive  viewpoint,  the  importance  of  establishing  mission  success 
as  a primary  design  goal,  and  techniques  for  assuring  implementation  of  this  goal. 


N ’Ielc* to  cliscuss: 


• -Design  considerations  and  responsibilities  for  Mission  Assurance  j 

• The  role  of  contractual  incentives  in  Management  Motivation  for  Mission  Assurance  j 

• The  role  of  Management  and  Program  Review  Process  in  Mission  Assurance^ 


• Techniques  for  management  and  control  of  the  development  and  application  of  systems  computer 


software^ 


• Subcontractor/supplier  interface  and  control  to  achieve  Mission  Assurance  j 

• Development  and  Control  of  microelectronics/hybrids  for  space  application  ; 


• Identification  and  examination  of  manufacturing  and  assurance  techniques  required  to  minimize  mission 
risk  j 


• Optimization  of  testing  of  components,  subsystems,  and  systems  for  Mission  Assurance  j 


• cu-i- 


• Industry's  approach  to  personnel  motivation  and  training  and  SAMSO/Aerospace/Industry 
experience  sharing,^ 


The  papers  prepared  by  the  Government  and  Industry  workshop  and  panel  members  reflect  advanced 
thinking  in  many  areas.  In  some  instances  the  positions  given  may  not  be  compatible  with  that  taken  by 
the  same  individuals  while  serving  in  their  official  capacity.  All  of  the  panel  members  were  encouraged 
to  divorce  their  "workshop  thinking"  from  the  constraints  imposed  by  existing  directives  and  practices. 
This  approach  was  considered  essential  to  the  effectiveness  of  the  conference.  The  papers  should  be 
treated  in  this  context. 
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INTRODUCTION 


Col.  William  L.  Bagwell 

Director  of  Launch  Vehicles  Procurement/SAMSO 
Gerald  E.  Lutz 

Director  of  Quality  Assurance 
Fairchild  Space  and  Electronics  Company 
(Chairman,  Air  Force  Liaison  Panel  — NSIA  QRAC) 

Good  Morning  • 

On  behalf  of  SAMSO  and  Industry  (NSIA,  AIA,  and  ADPA),  we 
wan:  to  welcome  you  to  what  might  well  be  an  historic  occasion 
an  exploration  and  open  exchange  of  technical  and  management 
information  intended  to  improve  SAMSO  Mission  Assurance. 

This  is  the  first  conference  known,  at  least  to  those  who  have 
worked  in  the  conference  framework,  that  has  crossed  discipline 
lines  in  a common  interest  and  objective  - very  refreshing  after 
years  of  trying  various  means  - Product  Assurance,  Mission 
Success,  Systems  Effectiveness,  etc. 

A real  keypoint  is  the  fact  that  Industry  and  Governmental 
agencies  have  found  a common  base  to  perform  a dialogue  for  a 
common  goal  - MISSION  ASSURANCE. 

A great  deal  of  credit  must  go  to  the  50  or  so  committee  members 
from  Industry,  SAMSO  and  Aerospace  for  their  dedication  in 
selecting  not  only  the  subject  matter,  but  by  designating  and 
allowing  top  people  in  the  field  to  present  their  points  of  view  in 
the  subject  areas. 

SAMSO  and  Aerospace  have  made  their  own  self-analysis,  now  it 
is  Industry's  opportunity  to  review  and  critique  SAMSO/ 
Aerospace  practices  in  a very  positive  professional  manner.  Our 
goals  within  each  session  are: 

• Identification  of  deterrents  and  counter  productive 
practices  having  an  effect  on  mission  attainment. 

• Development  of  cost  effective  approaches  to/or  better 
utilization  of  information  and  resources  to  enhance 
mission  success. 

• Determination  of  new  or  better  application  of  existing 
techniques  providing  a forcing  function  to  mission  risk 
minimization. 

Overall  objectives  of  the  conference  are  to: 

• Simulate  exchange  of  ideas  between  Industry  and 
Government  on  management/technical/acquisition 
support  aspects  of  mission  assurance. 

• Provide  a forum  through  which  the  total  community 
can  impact  and  address  problem  areas. 

• Provide  a continuance  to  the  SAMSO/Industry 
E xecutive  Sessions. 


Mission  Assurance  is  indeed  top  Management's  business, 
especially  if  the  lack  of  it  prevails  and  is  reflected  in  mission 
failures.  Failures  impact  follow-on  contracts  and  consideration  for 
other  future  business.  Management  motivation  can  be  achieved 
effectively  through  contractual  performance  incentives  properly 
timed  within  a contract.  In  a failure  mode,  schedule  and  other 
management/program  milestones  are  secondary  to  achieving  a 
successful  mission. 

Where  to  from  here?  A real  question  - should  we  continue 
Executive  Sessions  and  follow-on  conferences  or  workshops  in 
particular  interest  areas?  The  answer  to  this  lies  within  the  work- 
shop session  and  results  achieved.  An  aggressive  attitude  by 
management  and  all  disciplines  to  make  "success  happen"  is 
required.  Personally,  I would  like  to  entertain  an  Executive  Ses- 
sion in  six  months  to  review  any  impact  due  to  changes  imple- 
mented as  a result  of  either  the  earlier  Executive  or  these 
Conference  sessions. 


Please  be  aware  that  for  these  sessions,  the  Government  is  to  be 
treated  as  a partner  instead  of  a customer  — tell  each  other  your 
concerns.  It  is  incumbent  upon  Industry  to  define  counterpro- 
ductive and  cost/risk  drivers  through  analysis  to  assure  that  dollars 
spent  is  in  best  interest  of  both  Government  and  contractor 
relative  to  achievement  of  a complete  successful  mission.  There 
is  an  outgoing  challenge  to  Industry  to  make  both  good  and  bad 
experience  sharing  happen  so  that  all  parties  benefit.  If  we  are  to 
be  successful,  a free  and  open  exchange  is  essential.  We  encourage 
each  of  you  to  feel  free  to  express  your  thoughts  throughout  the 
workshops.  Workshop  leaders  will  present  aspects  of  their 
subject  matter  for  further  discussion.  These,  as  well  as  other 
recommendations  from  each  of  you,  are  the  backbone  of  the 
workshop. 
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WELCOME  ADDRESS 


Major  Goneral  Howard  E.  McCormick 
Vico  Commander,  SAMSO 


Good  moring,  ladies  and  gentlemen,  and  thank  you 
Mr.  Lutz  for  your  remarks. 

The  goals  and  objectives  established  for  this  con- 
ference are  both  vital  and  challenging.  On  behalf 
of  General  Morgan  and  SAMSO,  I want  to  thank 
industry  for  accepting  the  challenge  and  establish- 
ing this  forum  to  examine  the  important  aspects  of 
mission  assurance. 

General  Morgan  also  sends  his  personal  greetings 
and  welcome.  Although  his  retirement  this  week 
precludes  his  attendance,  he  sends  his  best  wishes 
for  a successful  and  productive  conference.  I 
should  add  that  General  Henry,  who  will  replace 
General  Morgan,  has  asked  me  to  express  his  sup- 
port for  the  objectives  of  the  conference  and  his 
best  wishes. 

During  the  joint  meetings  last  summer  with  industry 
executives,  many  of  whom  are  here  today.  General 
Morgan  expressed  the  SAMSO  position  that:  while 
the  record  of  our  spacecraft  and  missile  programs 
is  very  good,  the  impact  of  a small  number  of 
failures  can  be  devastating.  This  impact  goes 
beyond  the  initial  loss  of  operational  capability 
and  the  loss  of  resources.  It  also  seriously 
impedes  our  ability  to  obtain  congressional  support 
for  future  budget  allocations.  Management  at  all 
levels  has  to  recognize  that,  for  space  applica- 
tions, we  must  strive  to  achieve  perfection.  This 
may  be  initially  costly,  but  the  alternative  is 
even  more  costly  in  the  long  run. 

This  conference  in  itself  is  very  unique.  In  many 
conferences  you  find  a very  narrow  range  of  sub- 
ject matter,  and  a conference  of  experts  talking  to 
other  experts  on  the  same  subject  --  preaching  to 
the  choir  if  you  will.  Seldom  do  we  see  the  broad 
viewpoint  presented  in  a manner  that  cuts  across 
functional  lines  permitting  us  to  visualize  how  one 
area  or  function  affects  and  interacts  with  another. 
This  conference  certainly  provides  for  cross 
pollinization  and  "bridging  the  gaps"  among  func- 
tional lines. 

To  the  best  of  my  knowledge,  this  is  the  first 
conference  we  have  held  that  covers  so  wide  a range 
of  elements  which  are  key  factors  in  mission 
success.  This  multi-discipline  approach  - taking 
a horizontal  cut  at  the  entire  range  of  problems 
that  impinge  on  mission  success  - mark  this  as  a 
milestone  conference. 

No  one  realizes  more  clearly  than  this  assembled 
group  that  attaining  the  quality,  reliability  and 
performance  levels  required  for  our  space  and 
missile  systems  is  a tough  job  --  one  that 
requires  constant  attention  and  vigilance.  No 
single  segment  of  either  the  contractors'  organi- 
zations or  the  government  can  accomplish  these 
objectives  alone.  Working  as  a team,  we  must 
identify  and  adhere  to  certain  fundamentals  in  the 
acquisition  process. 

We  on  the  government  side  must  make  sure  that  we 
contract  properly  for  reliability  and  performance. 
This,  of  course,  must  include  proper  incentives  to 
achieve  a reliable  design  and  product. 

Contractors  must  strive  for  mission  assurance  from 
the  very  beginning  by  designing  systems  that 


eliminate  failuers  or  minimize  their  impact  if 
they  should  occur.  Constant  vigilance  must  insure 
that  reliability  and  quality  do  not  degrade  during 
manufacturing  and  efforts  must  be  sustained 
through  deployment,  launch  activities  and 
operations. 

And  we  can  do  it.  Our  government-contractor  team 
has  exhibited  many  examples  of  high  reliability 
and  exceptional  performance.  On  the  missile  side 
we  have  minuteman  III  gyro  accelerometers  which 
are  experiencing  a mean  time  between  failures  of 
150,000  hours  - which  is  over  17  years.  During 
that  time  the  gyro  makes  some  144  billion  revolu- 
tions. One  of  the  guidance  sets  that  uses  these 
gyros,  plus  15,000  other  piece  parts,  completed 
42,400  hours  or  almost  5 years  of  continuous 
running.  The  original  minuteman  I first  stage 
motors  have  a design  life  of  three  years,  and  we 
just  successfully  fired  one  that  was  cast  in 
June  of  '63,  almost  15  years  later. 

Our  satellite  programs  have  enjoyed  a similar 
record  of  success.  The  two  satellite  Nato  III 
communication  system  has  provided  an  availability 
rate  of  99.98  since  initial  launch  in  April  '76. 
The  first  of  our  latest  generation  weather 
satellites  is  presently  exceeding  its  15  month 
design  life  by  over  25%  and  is  still  going  strong. 

The  first  fleet  satellite  communications  system, 
our  most  complex  and  most  powerful  military  com- 
munications satellite,  was  launched  in  February, 
and  after  the  smoothest  pre  and  post  launch  check- 
out of  any  of  our  satellites,  was  declared  fully 
operational  one  week  before  its  scheduled  date. 

We  have  proven  that  we  can  be  highly  successful, 
and  that's  why  I am  pleased  with  the  positive 
aspects  of  the  title  of  this  conference  --  mission 
assurance.  We  must  not  dwell  on  the  negative 
aspects,  but  emphasize  success.  While  we  need  to 
identify  and  eliminate  those  pract  .es  and  techni- 
ques that  are  counter  productive,  we  must  also 
refine  and  continue  to  improve  upon  those  that  are 
tried  and  proven.  I uhge  you  in  your  workshops 
to  be  candid  in  your  observations  and  recommenda- 
tions for  improvements  in  our  directives  and 
manuals. 

Notwithstanding  the  many  directives,  regulations, 
manuals,  military  specifications,  and  all  of  the 
other  documentation  essential  to  our  business, 
ultimately  a successful  product  is  born  of  good 
people,  properly  organized,  skillfully  managing 
industry  and  government  resources.  In  today's 
environment  of  tightening  resources,  of  austere 
budgets  and  growing  concern  over  environmental 
impacts , reliability  of  our  products  is  no  longer 
a luxury.  We  cannot  trust  to  luck.  The  systems 
and  components  we  design,  develop- and  build  must 
work  --  the  first  timeT  Anything  else  is  unaccep- 
table  waste.  The  tools  to  assure  quality  of  pro- 
ducts are  available.  The  talented  people  who 
have  made  mission  success  a reality  in  many  of  our 
systems,  are  well  represented  in  this  conference. 

I am  certain  that  your  time  will  be  spent  to  good 
purpose  in  putting  tools  and  talents  together  for 
a new  level  of  achievement  in  mission  assurance. 

Thank  you  for  this  opportunity,  and  good  luck  in 
the  outcome  of  the  conference. 


I 


f 


A little  over  a year  ago.  It.  General  Thomas  W. 
Morgan  issued  what  was  in  effect  a challenge  to 
that  part  of  the  Aerospace  Industry  concerned  with 
SAMSO  projects  and  objectives.  His  challenge  was 
to  identify  and  suggest  solutions  to  general  pro- 
blems that  had  resulted  in  or  contributed  to  space 
mission  failures.  General  Morgan  was  concerned 
that  an  alarming  trend  was  developing.  His  con- 
cern was,  and  is  understandable.  SAMSO  missions 
are  vital  to  the  security  of  this  Nation.  A 
failure  can  leave  the  country  uncovered  and  even 
invite  trouble  from  the  outside.  Further,  such 
failures  are  very  expensive.  I am  sure  I do  not 
have  to  develop  this  point  further  with  this  audi- 
ence or  for  that  matter  with  any  taxpayer. 

During  mid  1977,  in  a series  of  meetings,  General 
Morgan  met  with  top  executives  of  industry  to  get 
their  points  of  view  on  enhancing  mission  assur- 
ance. I attended  one  of  those  meetings  and  have 
studied  a synopsis  of  all  of  the  discussions  and 
the  many  suggestions  brought  up.  I really  cannot, 
and  won't,  attempt  to  outline  what  I consider  to 
be  the  ultimate  solutions--that  is  really  a task 
that  this  conference  has  been  called  to  address. 

I would,  however,  like  to  highlight  several  parti- 
cularly interesting  areas  that  were  identified  in 
those  meetings  that  strike  me  as  being  fruitful 
for  much  discussion  during  this  conference. 
Specifical ly: 

1.  Design  - Key  Words--Overspecification; 
design  for  simplicity;  redundancy  with 
cross  strapping;  failure  modes  analysis. 

2.  Type  of  Contract,  Incentives,  and  Sub- 
sequent Relationships  - Key  Words--Fixed 
Price  vs.  C.P.F.F;  incentives;  type  of 
incentive;  arms  length  relationships  vs. 
over  involvement  on  the  part  of  the 
Government. 

3.  Parts  - Key  Words--Where  do  we  get 
reliable  parts  in  small  lots?  In  process 
inspection;  lot  sampling;  screening; 
burn-in. 

4.  Manufacturing  and  Quality  Assurance  - Key 
Words--Designing  in  inspectability;  con- 
figuration control;  transition  from 
design  to  fabrication;  continuity  of  the 
experienced  team;  role  of  quality 
assurance  personnel. 

5.  Testing  - Key  Words--Proper  environmental 
specification;  testing  at  lower  levels  of 
assembly;  realistic  simulation  of  on- 
orbit  operation;  analysis  of  test  data. 

6.  Experience  Sharing  - Key  Words--It's 
good  but  how  do  we  do  it?  Reporting  of 
both  flight  failure  and  anomaly  infor- 
mation as  well  as  success  stories, 
ALERTS?;  coupling  with  NASA. 


First  of  all,  the  bloom  is  off  of  the  space  rose. 
Time  was  when  space  projects  were  the  most  excit- 
ing and  important  efforts  in  the  country. 

Because  of  Russian  successes  and  our  early  fail- 
ures, the  very  prestige  of  the  United  States  was 
at  stake  with  its  own  people.  The  best  of  every- 
thing was  marshalled  into  play.  Nothing  was  too 
good.  The  satellite  projects  and  their  hardware 
were  treated  with  "tender  loving  care"  by  all 
concerned.  Citizens  and  their  organizations  sup- 
ported and  even  encouraged  greater  efforts.  In 
essence,  there  was  identi f ication  with  the  task  at 
all  levels  of  government,  industry,  and  the 
general  public.  Contrast  that  situation  to  where 
we  are  today.  From  the  public  standpoint  we  have 
been  to  the  Moon  and  space  exploration  is  now  too 
expensive  given  our  peoples'  perception  of  other 
needs.  Although  the  situation  may  be  changing  now, 
strong  anti-military  and  anti-technology  senti- 
ments have  developed  that  color  attitudes  and 
priorities.  Government  spending  is  under  pressure 
from  all  sides.  Inflation  is  a persistent  and 
unsolved  problem.  In  summary  on  this  point,  I am 
simply  saying  that  these  external  conditions,  which 
have  changed  so  dramatically,  have  reduced  motiva- 
tion markedly  and  driven  a number  of  small  busi- 
nesses and  even  a few  larger  firms  out  of  aerospace 
product  lines  completely.  How  many  times  have  we 
heard ‘the  comment  that  the  space  business  is  just 
not  predictable  or  profitable  enough  given  the 
investment  in  all  types  of  resources  reauired? 

A second  root  problem  is  that  spacecraft  are  not 
production  items  in  the  conventional  sense.  This 
is  the  "onesey"  or  "twosey"  problem.  There  is 
little  ability  to  operate  on  a learning  curve  thus 
increasing  both  reliability  and  efficiency  as  the 
serial  numbers  increase.  Further  there  is  no  room 
for  mistakes.  Space  systems  must  operate  with 
little  or  no  ability  to  correct  errors  from  the 
ground.  This  situation  creates  unique  technical 
and  managerial  problems  that  have  to  be  addressed 
in  special  ways,  many  of  them  unfortunately  based 
on  sad  experience.  But  this  can  result  in  quite 
a "hassle"  compared  to  other  lines  of  industrial 
endeavor  and  again  motivation  can  be  affected.  How 
many  times  have  we  heard  comments  to  the  effect 
that  extremely  tight  specification,  large  amounts 
of  paper  work,  onerous  terms  and  conditions,  etc., 
etc.,  are  just  not  worth  it  given  the  quantities 
involved? 

My  final  root  problem  (and  there  may  well  be 
others)  is  that  the  demands  we  are  making  on  our 
space  systems  are  resulting  in  more  and  more  com- 
plex spacecraft.  Aside  from  the  issue  of  whether 
the  extent  of  this  is  really  justified,  we  all 
know  that  system  reliability  is  the  product  of  all 
of  the  individual  part  and  component  reliabilities. 
The  more  parts  the  higher  their  reliabilities  must 
be  and  the  more  difficult  the  problem  becomes  from 
all  aspects.  Real  team  dedication  and  high 
motivation  are  required  on  all  of  our  parts. 


Now,  I would  like  to  make  some  personal  observa- 
tions concerning  what  I consider  to  be  some  root 
problems  in  our  industry  at  the  present  phase  of 
the  space  program  and  make  a recommendation  for 
improvement  in  a single  general  area  that  I feel 
bears  on  all  aspects  of  mission  assurance. 


Now  what  can  be  done  to  help  assure  mission 
success  given  the  foregoing  and  probably  many 
other  factors?  I suggest  that  a good  part  of  the 
ultimate  solution  lies  in  our  ability  to  increase 
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was  personally  impressed  with  the  point  of  view 
that  neither  the  "carrot"  nor  the  "kick-in-the- 
pants"  approach  would  really  ser  to  increase 
motivation.  What  is  really  needeo  is  personal  and 
even  organizational  identification  with  the  task 
at  hand.  This  is  the  so-called  "hobby"  theory 
that  holds  that  we  do  the  very  best  we  are  capable 
of  doing  when  we  are  working  at  a hobby.  Why? 
Because  for  some  reason,  or  reasons  we  want  to. 
There  are  seldom  tangible  rewards  and  rarely,  if 
ever,  is  punishment  involved.  We  become  deeply 
involved  and  motivated;  in  other  words  there  is  a 
high  degree  of  identification  with  the  task  and 
desire  to  see  it  completed  as  best  we  can. 

How  do  we  accomplish  this?  First,  some  conven- 
tional attitudes  must  change  both  towards  the 
invididual  and  towards  organizations.  We  have  to 
weld  our  engineers,  technicians,  and  assemblers 
into  teams  that  individuals  can  identify  with. 

They  must  understand  what  they  are  doing  and  why. 
They  must  feel  that  they  share  in  the  success  of 
the  project  and  that  each  of  their  efforts  is 
vital;  i.e.,  we  must  instill  a real  sense  of  pride 
of  accomplishment.  We  must  experiment  with  new 
concepts  such  as  giving  the  team  members  a greater 
say  in  their  destiny.  Communications  must  be  open 
and  direct.  Further,  those  of  us  at  the  higher 
levels  must  listen,  take  action  as  required,  and 
feed  back  the  results,  even  if  negative.  There 
are  many  small  things  that  can  be  done  that  we 
often  overlook  or  dismissed  as  "gimmicks".  For 
example,  project  insignia  that  can  be  put  on  work 
coats  or  clipped  on  jacke-s  cost  little  but  pay 
real  dividends.  Showing  films  of  launchings  or 
piping  real  time  count-down  audio  into  the  plant 
is  simple  but  very  popular  with  the  employee.  And 
don't  forget  to  keep  the  team  informed  of  how 
things  are  going  after  the  excitement  of  lift-off 
and  the  early  phase  of  mission  check-out  is  over. 
Do  they  know  what  was  really  accomplished  by  the 
mission  and  its  significance?  I know  classifica- 
tion is  a problem  but  much  can  be  done  if  we  put 
our  minds  to  it.  "Experience  sharing"  between 
aerospace  companies  and  between  -ndustry  and 
government  could  be  a useful  tool  in  developing 
new  ideas  for  increasing  motivation  and  for 
spreading  the  word  on  those  which  have  succeeded 
and  even  those  that  appear  to  be  dead  ended. 

These  are  only  a few  almost  random  ideas  on  the 
subject  that  only  scratch  the  surface.  The  point 
is  that  we  must  recognize  the  problem  and  do 
something  about  it.  Up  to  this  point,  1 have 
talked  mostly  about  individual  'ictivation,  but 
the  same  thesis  applies  to  whole  organizations. 
Here,  our  customer,  the  government  must  do  some 
re-thinking  about  organizational  motivation. 
Indeed,  I am  suggesting  that  the  "carrot"  (profit) 
or  "kick-in-the-pants"  (a  loss  on  the  job  and  no 
more  contracts)  approach  is  not  the  single  answer 
to  industry  motivation.  Corporate  pride  and 
image  are  important  factors.  Having  been  on  the 
industry  side  for  only  a year  now  It  would  be 
presumptuous  on  my  part  to  make  specific  sugges- 
tions in  this  connection,  but  I do  feel  that  it 
is  a fruitful  area  to  address. 

To  sum  it  up,  I feel  that  we  must  stress 
increasing  motivation  at  all  levels  because  of 
the  clear  benefit  to  many  aspects  of  the  problem 
of  increasing  mission  assurance.  Although  it  is 
only  a part  of  the  final  solution,  and  may  not 
be  easy  to  implement,  the  potential  pay-off  is 
large. 


General  Morgan,  you  have  challenged  industry 
to  help  you  solve  a serious  problem.  We  have 
responded  and  this  conference  is  one  important 
result.  I can  assure  you  that,  working  as  a 
team,  we  will  arrive  at  solutions  and  implement 
them.  Given  our  mutual  capabilities  and  desires 
any  other  alternative  is  unthinkable. 
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Introduction 

Ladies  and  gentlemen,  we  all  know  why  we're  here. 
We're  in  a very  public  bu  iness  that  demands  suc- 
cess of  a very  high  order  and  is  going  to  demand 
still  more.  It  is  perhaps  the  price  of  unparal- 
leled success  in  our  space  work  that  space  has 
evolved  from  the  exciting  to  the  unique  and 
essential.  Vital  questions  of  space  and  war  are 
decided  based  on  information  from  our  missions. 
Command  and  control  of  awesome  military  capability 
is  accomplished  increasingly  through  space 
elements. 

But  the  systems  we  provide  are  individually  large 
investments;  few  launches  cost  less  than  tens  of 
millions  and  some  missions  are  well  over  a hundred 
million.  At  those  prices,  we  don't  have  the 
luxury  of  warmup  tries  or  statistical  score  cards. 
Every  mission  must  work  as  close  to  100*  as  we 
can  achieve.  And  we  have  done  well  even  under 
these  conditions. 

But  the  future  holds  even  more  pressure  for 
mission  assurance.  We  are  entering  a period  in 
which  our  satellites  must  not  only  work,  they  must 
work  under  electronic  and  perhaps  physical  attack. 
Each  spacecraft  will  have  to  work  longer  --  as  one 
countermeasure  to  attack  we  need  to  accumulate 
numbers  of  spacecraft  on  orbit  at  a reasonable 
cost.  Our  spacecraft  will  have  to  work  more 
independently  of  ground  control  as  they  become 
more  complex,  as  they  are  put  in  space  storage, 
and  as  they  respond  intelligently  to  the 
unexpected. 

I've  been  asked  to  give  the  Government  Keynote 
Address  --  and  to  reiterate  the  charge  we've  been 
given  in  this  conference.  We  are  charged  with  the 
urgent  need  to  find  effective  solutions  for  the 
mission  assurance  problems  that  stand  in  the  way 
of  continued  success  in  the  future.  The  problems 
were  articulated  at  informal  meetings  General 
Thomas  W.  Morgan  held  last  fall  with  key  industry 
leaders.  Tomorrow's  workshops  are  organized 
around  the  results  of  those  meetings. 

Mission  Assurance  Techniques 

All  of  us  can  certify  that  to  achieve  our  levels 
of  reliable  performance,  we  must: 

• Define  it  in  the  beginning  --  success 
doesn’t  always  mean  the  same  thing  to 
everyone, 

• Design  for  it  early  --  even  in  our 
earliest  conceptual  thinking, 

• Test  for  it  as  each  subunit  is  fabri- 
cated, 

• Search  out  and  eliminate  each  failure 
mode  as  soon  as  discovered, 

• Control  the  configuration  with  a 
vengeance, 

• Document  even  the  apparently  trival 
anomaly, 

• Instrument  with  care  and  intelligence, 
and  then  -- 

• Do  it  all  over  again,  learning  from  our 
errors,  when  all  this  still  isn't 
enough. 


Since  this  won't  be  enough  in  the  future,  where  can 
we  look  for  further  advancements? 

Let  me  suggest  that  not  only  have  both  government 
and  industry  special  roles  to  play,  both  have 
special  obligations.  The  government  must  make 
timely  value  judgments,  define  mission  success  and 
stick  to  it,  and  accept  the  product  in  accordance 
with  pre-established  criteria  --  i.e.,  the  govern- 
ment customer  for  spacecraft  must  be  responsible 
and  well  informed.  The  industry  must  present  the 
government  with  internally  consistent  alternatives 
when  value  judgments  are  required,  alert  it  in  time 
for  decisions  when  needed  and,  of  course,  supply 
successful  spacecraft  and  launchers.  In  addition, 
each  party  must  hold  the  other  in  high  respect  -- 
troubles  at  the  interface  between  them  somehow 
always  show  up  in  reduced  spacecraft  performance 
on  orbit,  or  at  a minimum,  waste  of  precious 
resources. 

With  those  fundamentals,  long  since  accepted  by 
all  of  us  here,  let  me  start  what  I hope  will  be 
a good  list  of  constructive  suggestions  from  this 
conference. 


1.  The  government  should,  early  in  every 
program,  establish  the  order  of  the  com- 
peting characteristics  --  performance, 
cost,  schedule,  risk  and  reliability  and 
then  stick  to  it.  We  all  know  that  these 
characteristics  are  important,  but  what 
we  design  and  build  is  dramatically 
different  depending  on  their  relative 
priorities.  It  does  the  supplier  no 
good  at  all  to  say  that  all  character- 
istics are  equally  important;  worse  yet, 
it  sure  doesn't  help  to  change  the  order 
in  mid-program.  I was  lucky  when  I was 

a supplier  at  JPL  years  ago  to  have  a 
government  manager  who  did  set  the 
priorities.  Because  he  did,  we  together 
built  the  210-foot  antenna  at  Goldstone 
to  spec,  within  schedule,  and  to  cost  -- 
starting  with  assertions  from  others 
that  it  couldn't  be  done  at  all.  Time 
after  time,  when  troubles  showed  up,  the 
order  of  competing  characteristics 
pointed  the  direction.  The  character- 
istics, in  order,  happened  to  be  reliable 
performance,  cost,  and  then  schedule, 
each  having  both  a minimum  and  goal.  I 
wish  1 could  say  that  JPL  had  always  been 
so  lucky.  To  cite  a story  of  unrelieved 
misery  on  this  score,  read  the  recently 
published  history  of  Ranger.  To  bring 
the  lesson  learned  up  to  the  present,  we 
will  shortly  need  to  know  the  precedence 
of  survivabi 1 ity,  longevity,  cost  risk, 
and  Shuttle  compatibility,  relative  to 
the  characteristics  of  performance,  cost 
and  schedule  we  have  long  designed  and 
built  to.  It's  incumbent  on  the  govern- 
ment, of  course,  to  establish  the  order 
of  priorities.  It's  a difficult  task. 

And  it  has  a strong  impact  on  mission 
assurance.  Furthermore,  it's  even 
harder  to  stick  to  the  order,  particu- 
larly if  cost  is  not  first  in  priority, 
when  the  going  gets  rough. 

2.  The  government  and  industry,  together, 
need  to  recognize  reliability  budgets  for 
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what  they  are  --  a continuing  call  for 
assessment,  redesign  and  risk  apportion- 
ment --  and  not  just  a statistical  tool 
of  the  reliability  support  office.  When 
the  numbers  look  bad,  change  the  design! 
When  the  risks  finally  look  to  be  as 
balanced,  or  as  low,  as  possible,  use  the 
reliability  budget  to  design  the  test 
program  at  the  subunit  level.  When  all 
else  fails  to  produce  an  acceptable  reli- 
ability budget,  go  back  and  change  the 
requirements.  But,  above  all,  don't 
just  plow  blindly  ahead,  accepting  what- 
ever "the  state-of-the-art"  seems  to 
produce.  One  thing  about  reliability 
budgets,  as  the  program  proceeds,  they 
seldom  get  better  and  they  usually  get 
worse. 

The  industry  needs  to  help  the 
government  on  MILSPECS  --  that  hard- 
bought  repository  of  our  joint  corporate 
memory.  That  memory  is  remarkably  good 
--  commercial  industry  uses  it,  selec- 
tively, because  it  makes  for  reliable 
and  profitable  products  --  but  it  can  be 
faulty  or  obsolete.  Somehow,  we  need 
to  establish  an  environment  where 
industry  can  contribute  to  change  easily. 
The  DSB  Panel  on  Unreasonable  Specifica- 
tions found,  a few  years  ago, that  for  all 
the  frustrations  expressed  over  MILSPECS, 
very  little  practical  feedback  for  cor- 
rection was  occurring.  In  our  tight 
little  community  surely  we  can  and  must 
do  better.  Our  progress  in  the  evolution 
of  the  Class  S line  of  space  reliable 
piece  parts  shows  that  this  government/ 
industry  interchange  is  possible. 

I believe  the  government  should  provide 
considerably  more  publicity  for  the  con- 
tractors on  both  successes  and  failures. 
It  should  put  out  publicity  on  space- 
craft longevity  and  launch  successes  -- 
by  contractor  team.  It  is  my  strong 
impression,  going  back  30  years,  that  the 
contractors  get  all  too  little  public 
recognition  from  the  government  and  that 
this  lack  of  recognition  is  felt  by  the 
employees  who  do  the  work  that  results 
in  mission  assurance.  This  lack  is  too 
often  perceived  as  "the  government  has 
successes  and  the  industry  has  failures." 
That  perception  is  no  help.  But,  when 
the  government  does  its  public  relations 
well,  I've  observed  what  I call  "the 
supermarket  effect.”  This  beneficent 
effect  occurs  in  supermarkets  when 
employees'  spouses  are  greeted  by  their 
friends  with,  "I  see  that  TRW  (or  Hughes 
or  Rockwell  . . .)  just  put  up  another 
successful  satellite!  Was  your  husband 
in  that  project?"  The  joint  satisfaction 
of  the  spouse  and  the  employee  can  be 
pretty  strong.  Their  mutual  disappoint- 
ment at  failure  is  also  stronger  than 
that  of  the  employee  alone.  Thus,  more 
government  publicity  for  contractors 
should,  and  does,  work  to  the  govern- 
ment's advantage.  Remember  World  War  II 
when  everyone  knew  of  Martin  bombers. 


Lockheed  P-38’s  and  Kaiser  ships?  Do 
we,  in  this  gathering,  really  need  to 
remind  ourselves  that  we  succeed  or  fail 
together  and  that  we  should  stand  up  and 
be  counted,  together,  before  the 
country? 


5.  I'll  close  with  one  for  the  researchers. 
Our  community  is  beset  with  piece  part 
and  subcontractor  supply  problems.  We 
buy  only  small  quantities  of  things  on 
which  we  put  extraordinary  specifica- 
tions. To  the  suppliers,  there  is  often 
little  or  no  profit  in  it  and  seldom 
even  prestige.  What  more  can  we  do  about 
it?  We  have  already  tried  special  pro- 
duction lines,  special  QC  certification, 
special  testing  --  all  expensive.  So 
let  me  suggest  we  also  create  markets 
for  inherently  reliable  processes  much 
as  we  created  the  market  that  made 
integrated  circuits  a reality.  Accord- 
ing to  Bob  Noyce,  President  of  INTEL, 
transistors,  integrated  circuits  and 
microprocessors  were  developed  in  an 
almost  explosive  fashion  when  the  govern- 
ment created  a market  for  them  --  a 
market  rapidly  exceeded  by  a latent  com- 
mercial market.  If  you  look,  today,  at 
where  the  greatest  reliability  improve- 
ment has  been  realized,  by  a factor  of 
not  10  or  so  but  by  a factor  of  thous- 
ands. it  is  in  the  reliability  per  func- 
tion in  computing.  This  reliability 
improvement  came  from  new  processes 
permitting  more  functions  per  chip  on 
chips  of  increasing  perfection.  Of 
course,  the  temptation  is  to  use  the  new 
chips  to  carry  out  more  functions,  but 
we  might  do  much  better  to  trade  off  for 
more  reliability  of  a somewhat  lesser 
number  of  functions.  "Fault-tolerant" 
computers  do  just  that.  Solid  state 
electronics  is  now  increasingly  substi- 
tuting for  mechanical  elements  in  such 
mundane  products  as  typewriters, 
printers,  displays,  and  actuators.  Where 
else  can  we  repeat  the  process?  Laser 
fabrication?  Laser  propulsion?  Com- 
posite materials?  Solid  array  antennas? 
Remember,  the  trick  was  not  to  go  out  and 
fund  developments,  but  to  recognize  a new 
technique,  understand  its  system  signifi- 
cance, and  generate  the  market  for  it 
that  gets  private  industry  going. 


So,  there  are  some  ideas,  some  probably  better  than 
others.  No  doubt  we  will  get  together  a year 
hence  --  our  problems  won't  have  gone  away.  Per- 
haps we  should  review  our  ideas  from  this  con- 
ference then  and  see  if  and  where  we've  made 
progress. 


Good  Afternoon  ladies  and  Gentlemen 

Thank  you  for  inviting  me  to  share  some  of  my 
views  with  you  at  this  important  industry/SAMSO 
conference  on  mission  assurance.  You,  as  key  space 
industry  executives  play  a very  important  role  in 
what  I have  to  say. 

Where  you  sit  and  what  you  know  often  provide  you 
with  a different  perspective  than  you  might  other- 
wise have.  And  that's  true  with  respect  to  mission 
assurance.  One  of  my  favorite  stories  illustrates 
the  point.  Back  a few  years,  when  train  travel  was 
more  prevalent,  an  overbearing  company  president 
took  a business  trip  with  his  very  young  junior 
partner.  He  sent  the  young  man  ahead  to  get  a 
private  compartment  but  the  assistant  returned  with 
the  bad  news  that  the  best  he  could  do  was  share 
one.  When  the  president  arrived  at  the  shared 
compartment,  he  detected  a little  bit  of  col lusion 
in  that,  sitting  in  the  seat  facing  theirs  was  a 
lovely  young  lady.  She  was  accompanied  by  a 
matronly  woman  - perhaps  her  mother,  her  aunt, 
whatever  - but  obviously  a chaperone. 

The  two  businessmen  settled  in,  and  not  a word  was 
exchanged  as  the  train  pulled  out.  A short  way 
into  the  journey,  the  train  entered  a tunnel  and 
for  a minute  the  car  was  pitch  black.  In  that 
short  time,  there  were  two  distinct  sounds  - a 
kiss  and  a slap.  Then  the  train  was  back  into  the 
daylight,  and  the  foursome  resumed  their  stony 
silence.  Each  mul led  the  situation  over  in  his 
and  her  own  mind  - each  from  a different  position, 
and  with  different  knowledge. 

The  cute  young  thing  thought,  “Gee,  I was  hoping 
the  young  man  would  do  that,  he  certainly  is  a 
dashing  and  handsome  fellow,  but  I wish  Grandma 
hadn't  hit  him  so  hard." 

Grandma  was  fuming  to  herself,  "What  has  this  world 
come  to,  you  can't  even  ride  on  the  train  without 
some  masher  trying  to  take  advantage  - but  my 
granddaughter  took  care  of  him  - he  won't  ever  do 
that  again." 

The  old  president  was  reassessing  his  young  junior 
partner  - “Well,  maybe  he  is  a little  more  bold 
than  I thought  - maybe  he  wT 1 1 turn  out  to  be  the 
kind  of  aggressive  guy  I want  for  the  Chicago 
office  - but  wh.y~3io  the  young  lady  have  to  smack 
me?" 

And,  of  course,  the  junior  partner,  sitting  in  the 
right  spot  with  a_H  the  Information  was  saying  to 
himself,  "What  a great  day,  not  only  did  I get  to 
kiss  that  pretty  girl , I got  to  slap  the  old  man 
too." 

You  see,  where  you  sit  and  the  information  you  have 
give  you  a little  different  perspective.  For  the 
junior  partner,  his  planning;  design;  production, 
process  control;  motivational  incentives;  program 
or  situational  review:  and  experience  sharing  -- 
issues  of  your  mission  assurance  workshops  --  all 
can*  together  for  him  in  achieving  mission  assur- 
ance. His  mission  was  to  kiss  the  beautiful  young 
lady  and  to  Impress  the  old  man  - both  literally 
and  figuratively.  The  Air  Force's  objective,  in 
a broad  sense,  and  stated  very  simply,  is  to  pro- 
vide a service  - national  security  of  the  highest 
quality  - at  the  lowest  cost  and  to  assure  instant 
delivery  of  our  product. 


We  seek  the  same  goals  as  management  anywhere  - 
effectiveness  - which  we  more  often  characterize 
as  optimum  mission  performance  - and  economy  - 
minimizing  our  costs,  making  the  dollar  go  as  far 
as  possible.  We  seek  greater  efficiency  of  the 
fighting  man  through  training  and  through  more 
effective  weapon  systems  for  him  to  use  and  we  at 
the  same  time  are  striving  for  economy  - which 
has  always  been  a basic  concern  of  the  military  - 
in  fact,  a principle  throughout  the  history  of 
warfare  - described  by  such  phrases  as  economy  of 
force  - economy  of  effort  - economy  of  resources. 

This  brings  me  to  the  subject  of  mission  assurance. 
But  I want  to  talk  about  affordable  mission  assur- 
ance --  not  mission  assurance  at  an^  cost. 

With  the  imposed  discipline  of  the  new  Zero  Based 
Budgeting  system  and  prioritization  of  programs 
by  mission  categories,  we  must  think  and  act  in 
terms  of  affordable  mission  assurance.  If  not 
affordable  we  will  be  forced  to  seek  alternative 
means  to  achieve  that  mission  assurance.  In  my 
view,  the  B-l  is  a case  in  point. 

The  B-l  was  the  Air  Force's  number  one  priority 
weapon  system.  With  the  B-l  we  were  to  maintain 
mission  assurance  by  signif icantly  enhancing  the 
manned  bomber  leg  of  the  TRIAD  - with  a much 
stronger  and  survivable  strategic  deterrence  force. 

But  the  B-l  Production  Program  was  cancelled 
because  of  its  high  cost  --  not  only  acquisition 
cost  but  operating  and  support  cost  during  its 
life  cycle  as  well.  Now  the  Air  Force  is  embark- 
ing on  an  alternative  course  of  action  to  achieve 
that  measure  of  mission  assurance. 

So  I suggest  a change  in  your  assignment  --  and  its 
a much  tougher  assignment.  Instead  of  simply 
mission  assurance,  let  me  suggest  that  your  assign- 
ment be  "affordable"  mission  assurance.  Unless 
the  system  you  provide  --  and  these  are  the  sys- 
tems for  which  we  have  demonstrated  and  articu- 
lated a mission  need  --  unless  they  are  affordable 
we  will  not  be  able  to  achieve  the  scope  and  depth 
of  mission  assurances  we  really  need  for  our 
national  security. 

As  I reflect  on  the  fast-paced  changes  going  on  in 
the  Defense  acquisition  environment,  I am  con- 
vinced that  innovation,  imagination,  and  the  goals 
of  conferences  such  as  this  one  are  very  much 
needed,  if  we  are  to  successfully  meet  the  acquisi- 
tion challenges  presented  by  the  changing 
environment. 

Let's  look  at  the  nature  of  the  environment  in 
which  the  Department  of  Defense  must  accomplish  its 
mission.  Some  of  these  remarks  may  not  be  new  to 
you,  but  I believe  they  bear  repeating  because  they 
emphasize  the  seriousness  of  the  dilemma  we  face  in 
the  acquisition  business  --  they  establish  the 
framework  for  the  currently  emerging  acquisition 
policy  directions  in  the  DoD. 

We,  Government  and  Industry,  as  partners  in  the 
defense  acquisition  business,  are  working  toward  a 
common  goal  --  a U.S.  defense  capability  second  to 
none.  History  tells  us  that  the  acquisition 
environment  in  which  we  pursue  this  goal  is  one  of 
rapid,  sometimes  drastic  change.  Events  of  the 
past  decade  illustrate  this  point  vividly.  As  a 
nation,  we  have  gone  from  the  peak  of  the  conflict 
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in  Southeast  Asia  to  a time  of  peace.  We've  seen 
the  transition  from  the  draft  to  an  all -volunteer 
military  service.  We  have  witnessed  a shift  in 
national  priorities  and  goals. 

Internationally,  we  are  placing  more  reliance  on 
our  Allies  for  the  cooperative  defense  of  the 
free  world,  with  a trend  toward  more  industrial 
collaboration  for  the  acquisition  of  major  defense 
systems.  Domestically,  putting  men  on  the  moon 
and  ending  the  war  in  Southeast  Asia  have  been 
supplanted  by  environmental  protection,  social 
programs,  and  energy  concerns.  The  past  decade 
has  seen  general  economic  instability  - character- 
ized in  part  by  a depressed  stock  market  and 
spiraling  wages  and  prices.  We  have  also  been  con- 
fronted with  a dramatic  increase  in  fuel  prices  and 
fuel  shortages  that  were  unheard  of  ten  years  ago. 
All  of  us  have  been  talking  about  new  highs  for 
both  our  private  and  corporate  utility  bills  and 
wondering  how  high  they  will  continue  to  rise. 

All  of  these  changes  have  had  a great  impact  on  the 
economics  of  the  Department  of  Defense.  This  impact 
is  perhaps  best  reflected  in  the  DoD  budget  trends. 
The  proposed  DoD  FY  79  budget  outlays  are  large  in 
terms  of  current  dollars,  but  small  in  other  res- 
pects. A decade  ago  in  FY  68.  DoD's  budget  outlays 
accounted  for  9.4  of  the  gross  national  product 
and  43.6  of  the  federal  budget.  The  FY  79  budget 
is  estimated  to  prepresent  5.1  of  the  GNP  and  23 
of  the  federal  budget.  Since  1968  we've  seen  the 
budget  grow  in  current  dollars  from  $78  billion  to 
its  FY  79  estimate  of  $115  billion.  At  first 
glance  - a 47'  increase.  However,  converted  to 
constant  1979  dollars,  we  see  a decl ine  from  $165.8 
billion  in  1968  to  the  present  $115  billion  level, 
a 30%  decrease. 

During  this  period  of  declining  economic  strength, 
other  trends  were  also  affecting  DoD.  For  example, 
from  1964  to  1977,  the  total  manpower  strength  of 
the  Air  Force  declined  from  about  1,200,000  to 
815,000  - 32%  decrease.  During  the  same  period, 
manpower  costs  went  up  almost  80  . The  cost  of 
operating  and  supporting  existing  weapons  systems 
was  increasing  rapidly.  The  cost  of  acquiring 
new  systems  was  increasing  rapidly.  And  with 
military  programs,  cost  growth  became  a house- 
hold word. 

Let  me  be  more  specific.  The  Air  Force  recently 
submitted  its  calendar  year  1977  report  to  Congress 
for  selected  major  acquisition  programs.  For  12 
major  Air  Force  programs,  excluding  the  cancelled 
B-l,  we  experience  a 6%  increase  for  1977  - total- 
ing over  $2.8  billion  --  in  one  year.  And  1977 
was  a good  year  for  the  Air  Force.  This  net 
increase  represents  an  aggregate  of  increases  and 
decreases  over  an  entire  range  of  cost  categories 
for  the  12  programs.  We  experienced  increases 
in  these  categories:  Inflation  - over  $2.5  billion; 
Quantity  Changes  - over  $250  million;  Schedule 
changes  - over  $850  million;  Engineering  - over 
$400  million;  Support  - over  $50  million  Contract 
overruns  and  others  - about  $117  million.  These 
were  offset  by  estimating  refinements  that  reduced 
costs  by  about  $1.4  billion.  These  figures  tell 
us  that  Inflation  Impacts  program  costs,  and  that's 
no  surprise.  But  they  reveal  that  almost  every 
other  category  of  cost  Is  also  susceptible  to 
growth.  No  part  of  the  acquisition  formula  has 
been  unaffected. 
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In  short,  the  nature  of  the  defense  acquisition 
environment  could  be  described  as  one  of  declining 
real-term  budget  in  the  face  of  increasing  weapon 
systems  acquisition  and  operating  costs.  This 
dilemma  embodies  the  challenge  for  improving 
defense  acquisition  and  mission  assurance.  A 
number  of  actions  are  needed  if  the  defense 
acquisition  process  is  to  remain  responsive  and 
efficient  in  the  face  of  this  dilemma.  Many 
significant  actions  are  already  underway. 

At  the  legislative  level,  Senator  Chiles  has 
introduced  Senate  Bill  1264,  the  Federal  Acquisi- 
tion Act,  which  if  enacted  will  form  a new 
statutory  base  for  acquisition. 

The  Office  of  Federal  Procurement  Policy  has 
emerged  as  a powerful  force  for  change  in  the 
acquisition  process.  With  the  backing  of  both  the 
Congress  and  the  Office  of  Management  and  Budget, 
0FPP  has  given  high  priority  to  implementation  of 
0MB  Circular  A- 1 09 . A- 1 09  requires  us  to  express 
our  needs  in  terms  of  the  mission  we  are  trying 
to  satisfy  rather  than  with  a particular  hardware 
design  with  specific  performance  characteristics. 
It  also  emphasizes  the  competitive  exploration  of 
alternative  system  design  concepts  in  response  to 
mission  needs.  DoD  has  implemented  this  policy 
and  the  Air  Force  has  promulgated  it  to  the  field. 
The  Air  Force  submitted  its  FY  79  RDT&E  budget  on 
the  basis  of  mission  need.  In  the  future  we 
expect  our  entire  budget  to  be  submitted  on  that 
basis. 

0FPP  has  also  started  the  effort  to  develop  a 
single  uniform  acquisition  regulation  for  the 
federal  government  --  the  "FAR"  or  Federal 
Acquisition  Regulation.  The  final  regulation  is 
to  be  published  in  August  1979  and  will  have  far 
reaching  effects  on  the  acquisition  process. 


Within  the  Department  of  Defense  a major  change  has 
taken  place  with  the  merger  of  the  research  and 
engineering  function  and  the  procurement  function. 
This  is  part  of  an  effort  to  streamline  the  organi- 
zation and  focus  attention  on  the  total  acquisition 
process  under  the  acquisition  executive.  This  is 
a significant  step  away  from  our  earlier  tendency 
to  make  R&D,  systems,  and  follow-on  buy  decisions 
in  a parochial  vacuum  --  on  a compartment  by  com- 
partment basis.  Believe  me,  this  is  not  a 
cosmetic  reorganization.  It  is  intended  that 
there  be  a basic  change  in  our  acquisition 
philosophy.  Historically  the  primary  focus  of 
"contracting"  people  has  been  on  the  negotiation  of 
cost  or  price  which  likely  has  little  relationship 
to  the  total  cost  to  the  Government  and  is  only  a 
small  part  of  the  acquisition  professional's  role. 
There  is  now  the  need  for  "acquisition  managers" 
who  think  and  understand  the  broad  spectrum  of 
acquisition. 

There  are,  of  course,  many  more  initiatives  that 
will  affect  the  acquisition  business.  But  let  me 
stop  with  these  and  reflect  a moment  on  what  these 
changes  say  to  me  and  where  I believe  we  need  to 
focus  our  attention.  I think  the  direction  is 
clear.  The  initiatives  and  changes  I mentioned  are 
all  working  to  sharpen  our  focus  on  the  comprehen- 
sive management  of  the  total  weapon  acquisition 
process.  Decreasing  buying  power,  increasing 
weapon  costs,  shifts  in  national  priorities,  and 


8 


V 


1 


common  sense  have  brought  us  to  the  point.  These 
trends  together  with  the  emphasis  on  the  total 
acquisition  process  embody  the  challenge  for 
mission  assurance. 

In  order  to  meet  this  challenge,  we  must  take 
action  in  several  areas.  First,  the  broad  concept 
of  the  total  acquisition  process  demands  that  we 
place  greater  emphasis  on  the  application  of 
acquisition  techniques  that  foster  interdiscipl inary 
cooperation  in  sort  of  a symbiotic  way.  The 
techniques  that  fall  under  the  life  cycle  cost 
umbrella  are  good  starting  points.  In  the  past  few 
years  we  have  begun  shifting  from  a design-to-unit- 
cost  perspective  to  a design-to-life-cycle-cost 
perspective.  This  approach  considers  the  sum  of 
acquisition,  support,  and  operating  costs  and 
demands  that  maintenance  and  logistics  considera- 
tions be  c nsidered  during  the  initial  engineerino 
design,  (}  even  in  space  programs).  We  must 
work  together  to  refine  the  application  of  the  life 
cycle  cost  techniques  with  the  goal  in  mind  of 
making  early  attractive  rewards  possible  for  con- 
tractors who  can  effectively  control  and  reduce 
systems  life  cycle  costs  by  enhanced  reliability 
and  maintainability. 

Second,  we  must  develop  closer  ties  between 
program  cost  estimating  and  contract  pricing. 
Frequently  our  program  budget  cost  estimates  are 
based  on  financial  data  that  bear  1 ittle  resem- 
blance to  equivalent  data  generated  by  contract 
pricing.  Each  set  of  data  is  based  on  different 
assumptions  and  different  computation  techniques. 

It  is  little  wonder  that  actual  program  costs 
frequently  don't  match  the  original  estimates.  I 
do  not  pretend  that  the  marriage  of  budgetary  and 
contract  estimating  and  pricing  will  get  rid  of 
overruns,  but  it  will  certainly  contribute  to 
better  mutual  understanding  of  the  two  processes 
and  enable  us  to  better  analyze  and  explain 
variances  and  to  make  the  appropriate  program 
decisions.  I urge  you  to  exhange  ideas  and 
thoughts  in  this  important  undertaking. 

Third,  if  we  are  to  successfully  deal  with  cost 
growth  and  overrhos,  and  avoid  the  problems 
associated  with  "buy-ins"  we  must  work  for  more 
cost  realism  in  proposal  evaluation  and  source 
selection.  This  translates  into  increased 
emphasis  on  and  better  use  of  most  probable  cost 
estimates.  We  must  go  beyond  merely  determining 
that  proposed  prices  are  fair  and  reasonable-and 
ask  the  question:  "What  is  the  most  likely  over- 
all cost  to  the  government  if  a contract  is 
awarded  to  a particular  offeror?"  That  question 
can  embrace  development,  acquisition,  and  operating 
and  support  costs,  or  any  subset  of  them  depending 
on  the  contracting  situation.  This  approach 
places  two  important  responsibilities  on  the 
government.  First  we  must  develop  accurate 
estimates  of  most  probable  costs.  And  second, 
we  must  fairly  and  effectively  apply  the  most 
probable  cost  estimate  as  an  evaluation  factor 
based  on  the  considered  assessment  of  a proposal's 
technical  deficiencies  and  the  cost  delta  to  be 
applied  to  that  proposal  in  assessing  the  antici- 
pated cost  of  performance. 

This  afternoon  I have  tried  to  build  a perspective 
around  the  direction  I see  the  defense  acquisition 
business  heading  and  the  need  for  affordable 
mission  assurance.  The  trends  of  the  past  decade 


have  created  a dilemma  involving  austere  budget 
constraints  and  increasing  defense  acquisition  and 
support  costs.  A number  of  legislative  and  policy 
changes  have  clearly  mapped  out  the  direction  for 
the  future  --  better  management  of  the  total 
acquisition  process. 

I think  the  bottom  line  for  us  at  this  conference, 
even  if  we,  like  the  junior  partner,  work  in  the 
dark  sometimes  - we  get  the  job  done,  because  we 
have  the  right  perspective  - we  sit  in  the  right 
seat  and  have  the  right  information.  It  is 
important  to  our  nation  that  we  continue  that  way-- 
and  it  is  likewise  important  that  industry  under- 
stands our  perspective  so  that  you  can  help  us. 

I think  you  in  industry  should  have  a simple  time 
of  that  - our  perspective  is  so  much  like  your 
onw.  I solicit  your  continued  support  and 
enthusiasm  in  our  efforts  to  maintain  a defense 
that  is  second  to  none. 

Thank  you. 
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Presentation  by  Mr.  Lowrie 


MR.  LOWRIE:  I really  don't  know  why  you're  having 
this  symposium.  I was  down  in  the  Virgin  Islands 
last  week  and  and  the  privilege  of  flying  in  a 
DC-3.  I hadn't  been  in  a DC-3  for  so  many  years, 

I forgot  what  they  were  like.  But  any  country  that 
could  produce  a DC-3  in  the  '40s  which  is  still 
flying  in  the  Virgin  Islands,  with  about  half  the 
rivets  replaced  with  bolts  that  are  sticking  up 
into  the  windstream  and  gaps  everywhere  between 
panels  where  we  aerodynamicists  used  to  be  so 
fussy,  doesn't  need  to  worry  about  Mission  Success. 
It  gave  me  a nightmare.  I didn't  think  we'd  ever 
get  off  the  ground  in  something  as  patched  up  as 
that,  but  we  made  it. 

Seriously,  for  the  next  half  hour,  IV  like  to 
share  with  you  some  of  my  observations.  I'd  like 
to  start  with  a recap  on  our  heritage  in  the  mis- 
sion success,  mission  assurance  business,  from  my 
point  of  view,  as  far  as  history  or  where  you've 
been.  Then  I'd  like  to  reflect  on  the  Implications 
on  what  I call  attitudes  towards  mission  assurance 
or  maybe  attitudes  that  are  detrimental  to  mission 
assurance  and  what  I have  referred  to  as  some  of 
the  "myths"  of  the  business. 

I won't  go  clear  back  to  the  Wright  Brothers  or 
Goddard.  I'd  like  to  start  in  the  mid  50 ' s and 
say  where  we've  come  from,  from  that  time.  That  is 
the  start  of  the  ICBM  business.  I can  reflect  back 
on  many  meetings  we  had  in  those  early  days  on 
Titan  I.  There  were  bitter  debates  or  maybe  I 
should  say  active  debates  between  some  of  the  RW 
people  and  ourselves.  For  those  of  you  who  don't 
have  the  memory,  that's  kind  of  a forerunner  of 
Aerospace  Corporation  in  generations  back.  These 
were  debates  on  how  many  flight  failures  we  should 
allow  for  during  the  flight  test  program.  You 
should  reflect  on  that  a little  bit  because  we  were 
arguing  whether  it  should  be  15  percent  or  20  per- 
cent or  25  percent  failures  in  flight.  That's  the 
start  of  the  business.  The  implication  is  very 
profound,  because  if  your  expectation  is  failure, 
you  have  to  have  an  attitude  of  condoning  failure. 
So  that's  where  we  started  from  20-some  years  ago-- 
expectation  of  15,  20,  25  percent  failure. 

I'm  not  going  to  give  you  a big  long  history 
dissertation.  I'll  jump  next  to  the  early  60's  and 
the  start  of  Titan  III.  The  first  Titan  III  pro- 
gram was  a 17-ship  R&D  program.  Titan  III  was 
originally  conceived  as  a core  vehicle,  which  we 
called  the  Lot  A vehicle  on  which  you  could  fly 
small  payloads  to  various  orbits.  Then  you 
strapped  on  a pair  of  solids  and  you  could  fly  a 
bunch  of  other  tougher  missions.  The  17-ship 
R&D  program  checked  out  the  different  missions  and 
the  different  flight  profiles  gradually  working  up 
to  tougher  and  tougher  missions.  There  was  a kind 
of  underlying  implication  that  there  would  pro- 
bably be  some  failures  along  the  way,  because  we 
weren't  carrying  any  active  payloads.  We  were 
carrying  all  dummy  payloads  initially. 

I think  we  had  the  successes  in  a row  and  boy, 
everybody  got  excited  and  the  forerunner  of  SAMSO, 


I don't  remember  whether  it  was  BMD  or  BSD  at  that 
time  said,  "Gee  whiz,  we're  spending  a lot  af  money 
flying  boosters.  They're  awful  damn  fine  vehicles 
and  we  ought  to  put  some  bonus  payloads  on  them." 
That  seemed  like  a jim-dandy  idea  and  so  they  went 
around  to  industry  and  said,  "Hey,  you  know,  we  got 
a nice  vehicle  that  you  can  get  into  space  with 
and  you  ought  to  come  join  the  party;  but  be  aware 
it's  a bonus  payload  because  it's  an  R&D  program 
and  we  can't  guarantee  you  anything."  So  we 
started  flying  some  pretty  nice  little  payloads. 

Then  the  stuff  started  to  hit  the  fan,  because  we 
had  two  failures  in  a row  and  dumped  two  payloads 
in  the  ocean.  They  were  no't  the  big  kind  of  pay- 
loads  you  have  now-a-days,  but  still,  it  got  every- 
body in  the  system  up  tight. 

I'll  never  forget  those  times.  I had  just  been 
appointed  the  glorious  title  of  Director  of 
Mission  Success. 

My  general  manager  said  that  my  function  in  life 
was  to  be  damn  sure  we  never  dropped  anything  in 
the  ocean  from  now  on.  Of  course,  he  didn't  count 
those  against  me,  because  that  kind  of  happened 
before  my  watch.  At  that  point  SAMSO  or  its  fore- 
runner called  the  Associate  Contractor  out  to  the 
West  Coast  to  a kind  of  a come-to-God  meeting.  I 
think  it  was  Colonel  Miller  at  that  time.  He 
lambasted  us  all  and  said,  "you've  all  done  a lousy 
goddamn  job",  and  he  had  a bill  of  particulars  that 
they'd  gone  off  and  gotten  on  what  a lousy  job  we 
had  really  done  --  all  of  us.  Then  he  aaid  we've 
got  to  be  100  percent  successful  now  that  we're 
flying  expensive  payloads. 

"I  want  you  all  to  go  home  for  ten  days,  and  then 
come  back  and  tell  us  what  you're  going  to  do 
about  it."  I'll  never  forget,  my  general  manager 
and  the  program  manager  of 'Titan  III  and  I got 
drunk  on  the  airplane  going  home  and  mumbling  into 
our  beer,  "Those  bastards,  they  contracted  for  an 
80  percent  reliable  product  and  we're  giving  them 
a damn  fine  product."  After  we  got  rid  of  all  that 
venom,  cleaning  the  spleen,  or  whatever  you  want 
to  call  it,  the  next  day  we  sat  down  and  got  a 
little  more  mature  about  it.  We  decided  we  really 
ought  to  take  the  challenge  seriously  and  so  did  the 
other  associate  contractors.  That's  really  the 
turning  point,  at  least  in  our  company,  and  maybe 
even  in  much  of  the  industry,  of  some  of  the  basic 
attitudes.  I'll  go  into  some  of  those  attitudes 
in  a minute. 

We  did  then  go  through  a soul-searching  exercise 
and  a moratorium  for  about  a month  or  six  weeks. 

We  went  back  through  all  of  our  test  procedures, 
did  a kind  of  a fault  tree  analysis  from  a testing 
point  of  view.  We  found  lots  of  gaps  where  we 
could  test,  should  test  and  were  not  testing, 
because  we  really  hadn't  done  the  original  job 
thoroughly  enough.  We  went  back  and  looked  at  all 
of  our  failure  histories  and  we  were  appalled  at 
what  we  saw  in  those  fine  documents.  You  all  have 
a spec  that  says  you  have  to  have  a corrective 
action  system.  We  did.  We  had  all  sorts  of  good 
records,  but  when  you  went  and  looked  and  you  tried 
to  find  out  what  was  the  physics  of  the  failure  -- 
did  we  really  isolate  the  failure  --  you  came 
away,  in  many  cases,  scratching  your  head.  We  went 
and  looked  at  the  paperwork  that  accepts  the  hard- 
ware, and  came  away  building  a cage  for  rejected 
hardware  because  the  paperwork  wouldn't  support 
its  acceptability.  You  didn't  know  whether  it  was 
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a paperwork  problem  or  a real  hardware  problem.  In 
many  cases,  we  went  back  to  the  vendors  and  got  the 
primary  data,  test  dates  or  other  things  and  found 
out,  yes  it  was,  in  fact,  good  hardware  and  people 
were  just  sloppy  with  the  paperwork.  In  other 
cases,  we  had  to  retest  hardware.  In  still  other 
cases,  we  found  problems  and  rejected  hardware.  It 
was  a real  gut  wrenching  exercise.  We  found  out, 
down  in  the  guts  of  our  operation,  we  weren't 
doing  nearly  as  good  a job  as  we  thought  we  were. 

Moving  ahead  in  history  a little  bit  later,  John 
Naugle  of  NASA  asked  a couple  of  us  to  come  in  and 
talk  to  him.  He'd  had  two  or  three  failures  on  the 
Delta  program.  He  was  pretty  concerned  about  what 
he  should  do.  Coincidently,  I'd  been  to  a Delta 
Users  symposium  at  Goddard  two  weeks  earlier 
because  I was  about  to  go  bid  on  a thing  called 
Planetary  Explorer.  This  Planetary  Explorer  was 
one-of-a-kind  spacecraft.  I was  sitting  in  the 
audience  as  a potential  user  and  the  man  on  the 
stage  was  very  blantantly  telling  me,  "You're  a 
user,  if  you  are  going  to  fly  four  or  five  pay- 
loads,  you  ought  to  expect  one  going  in  the  ocean, 
because  this  is  an  80  percent  reliable  booster  and 
therefore  you  should  expect  failure."  That  kind  of 
wrenched  my  gut  a little  bit  as  a user.  So  any- 
way, when  we  were  talking  to  John  we  talked  all  the 
good  things  they  were  going  to  do  in  their  shops, 
with  their  parts,  with  their  testing  and  all  that 
kind  of  thing.  At  one  point  I turned  to  John  and 
said,  "You  know  what  your  primary  problem  is?" 

He  paused  and  I said,  "It's  you!"  I knew  John 
pretty  well  and  1 wanted  to  shock  him.  1 said, 

"You  sat  in  the  audience  with  me  and  you  let  some 
of  your  people  in  the  lower  levels  of  your  organi- 
zation condone  failure.  Management  can't  condone 
failure."  Now,  that  is  not  an  unrealistic  state- 
ment. Failure  is  going  to  occur.  I understand 
that.  I've  lived  in  the  real  world,  just  like  the 
rest  of  you,  but  management  can  never  put  them- 
selves in  a position  in  our  kind  of  business  of 
condoning  failure  or  saying,  "1  expect  a 20  per- 
cent fallout."  But  that's  where  our  heritage  has 
been  in  this  industry  and  we're  not  out  of  it  yet. 


I don't  want  to  put  myself  in  the  position  of 
crowing,  but  I would  like  to  talk  Viking  a little 
bit,  to  illustrate  that  the  industry  has  matured. 
We've  come  an  awfully  long  way.  In  ! 969  Martin 
Marietta  contracted  for  Viking  with  NASA  and  over 
half  of  the  fee  of  that  contract  was  to  be  an 
objective  award  fee  by  NASA  based  on  the  return  of 
science  data  from  Mars.  Now  that’s  a pretty  gutsy 
proposition.  You're  betting  a lot  of  money  and  a 
lot  of  corporate  investment  on  being  successful  and 
delivering  the  merchandise  on  a one-of-a-kind  type 
program.  We're  not  alone.  There  are  other  com- 
panies that  are  doing  the  same  sort  of  things. 

There  are  a number  of  satellite  programs  now  that 
are  required  to  live  in  space  for  many  years  and 
produce  part  of  their  fees  from  Incentive  awards 
based  on  performance.  We  have  come  a long  way  and 
we  have  demonstrated,  as  an  Industry,  that  we  can 
do  things  like  Viking.  On  that  program  we  had 
14  major  subcontractors  --  many  of  them  very  big 
companies  in  the  Industry  --  some  of  them  I see 
here  today.  All  of  them  pulling  a big  chunk  of 
the  weight  that  produced  Viking.  So  the  Industry 
can  produce  that  kind  of  results.  I went  through 
that  history  because  I think  sometimes  it  is 
Important  to  understand  where  you've  come  from  to 
understand  maybe  where  it  is  you've  got  to  go.  I 
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want  to  talk  now  about  some  of  the  myths  and 
attitudes  I've  seen  along  the  way. 

The  first  thing  is  that  question  of  attitude  toward 
failure.  I think  it's  absolutely  fundamental  and 
I think  you  all  have  to  look  in  the  mirror  and 
say,  "Are  we  guilty  of  expecting  failure  and  there- 
fore passing  on  down  through  the  organization,  an 
image  of  condoning  failure?"  Because  that  reflects 
very  much  in  the  performance  of  the  total 
organization. 

I have  two  or  three  other  myths  I want  to  go  over 
in  the  time  available.  I'm  going  to  the  trenches-- 
down  to  the  working  level  of  the  mission  assurance 
business.  The  first  myth  I call  the  personnel 
error  myth.  If  I go  back  ten  or  fifteen  years, 
it  was  rampant  throughout  the  industry,  but  it  is 
still  pretty  prevalent.  You  go  down  in  your 
organization  and  look  at  your  failure  records. 
Examine  them.  You'll  probably  find  many  cases 
where  it  says,  "Cause  of  failure,  personnel  error-- 
corrective  Action,  personnel  warned."  That’s 
phoney  in  most  cases.  The  personnel  error  myth  or 
whatever  you  want  to  call  it,  is  a trap  because 
it's  a garbage  can  for  hiding  all  sorts  of  other 
things.  Personnel  do  make  mistakes,  and  I'm  not 
against  motivation  programs  to  minimize  that,  I 
think  we  would  have  them.  They're  great,  particu- 
larly to  get  management  and  supervision  to  talk  to 
the  worker.  That  gets  him  motivated,  but  it  gets 
him  motivated  because  he  begins  to  understand 
what's  going  on  and  what  is  expected. 

However,  there  are  many,  many  cases  - and  I'll  just 
use  one  as  an  example  out  of  our  own  company  -- 
of  so-called  personnel  error  that  aren't  that  at 
all.  They're  supervision  error  or  management 
error.  The  example  I'm  going  to  use  is  a few  years 
back,  but  we  can  find  them  every  day.  I got  a call 
one  day  from  the  Cape  that  said,  "Gee,  one  of  our 
bumble-headed  workers  put  a foot  over  on  the  side 
of  the  boat  tail  and  knocked  off  a transducer;  now 
we  got  to  delay  the  launch  for  two  days,  because 
we  can’t  get  the  thing  replaced."  Well,  I knew  the 
boat  tail  on  a Titan  is  a cramped  place  to  work  so 
when  the  general  manager  came  running  in  my  office 
and  said,  "Have  you  fired  him  or  given  him  five 
days  off?"  I said,  "We're  investigating."  "What 
do  you  mean  you're  investigating?  He  kicked  the 
transducer."  Well,  the  investigation  showed  that, 
as  1 said,  the  guy  has  to  crawl  in  a tight  area  and 
he  has  got  to  get  into  a posture  where  he  is  reach- 
ing up  high.  We  didn't  give  him  a work  platform 
and  we  had,  for  several  birds,  been  allowing  the 
the  workmen  to  slide  a two-by-six  across  two  frames 
and  then  crawl  in  on  the  two-by-six,  stand  erect 
and  brace  himself.  In  order  to  keep  his  balance 
he  had  to  put  his  foot  on  the  side  of  the  launch 
vehicle.  That  was  the  work  environment  we  gave 
that  poor  son-of-a-gun  to  work  in.  Now  whose  fault 
is  it  the  transducer  got  kicked?  It's  obviously 
management  or  supervision. 

The  supervisor  had  been  watching  that  go  on  and 
hadn't  screamed.  If  you  carefully  examine  so- 
called  personnel  error,  in  almost  all  cases, 
you'll  find  somebody  needs  some  training,  or 
education,  or  a better  tool,  or  a better  work 
environment  or  something  else.  He  doesn't  need  to 
be  cautioned,  in  most  cases.  The  thing  I would 
challenge  you  to  go  do,  is  look  at  your  various 
companies  and  see  if  you're  using  personnel  error 
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as  a trap  for  getting  around  working  to  correct 
the  real  problems. 

The  next  myth  I want  to  talk  about  is  the  corrective 
action  system  itself.  Everyone  has  a closed  loop 
corrective  action  system.  Every  company  I've  ever 
gone  into  in  the  United  States  beats  their  breast, 
shows  a viewgraph  of  their  closed  loop  corrective 
action  system.  Of  course,  they  have  one,  the 
specs  require  that  they  have  them,  but  if  you  go 
down  in  the  bowels  and  examine  how  they  run  it,  you 
find  sometimes  it's  pretty  shoddy  in  the  quality 
of  implementation.  So  I say  it's  a myth  in  the 
sense  of  the  quality  of  the  implementation. 

One  of  the  things  we  started  back  in  the  Titan  III 
trauma  days  was  an  education  of  our  engineers  and 
testers  and  quality  engineers,  to  treat  all  fail- 
ures with  the  same  rigor  as  if  it  was  a flight 
failure.  That  requires  that  you  must  get  to  the 
physics  of  the  failure.  You  must  isolate  the 
fault  and  must  take  corrective  action  or  bump  it 
up  to  the  project  manager.  That  way  no  problems 
are  hiding  down  there  in  the  woodwork. 

The  thing  I find  that  we,  as  an  industry,  still 
have  to  go  a long  ways  to  do  better  in  this  con- 
cept of  cause  isolation  from  a system  level  clear 
down  to  the  parts  level  into  the  physics  of  fail- 
ure. Let  me  illustrate  what  I'm  talking  about. 

If  I have  a black  box  failure  in  the  shop  and  I 
isolate  the  problem  to  a module  and  I take  the 
module  out,  and  then  put  a new  module  in  and  I go 
test  the  box  and  it  seems  to  work  great,  there  are 
still  a lot  of  people  in  the  industry  saying,  "Aha, 
that's  a good  piece  of  merchandise."  That  isn't 
necessarily  true.  The  first  thing  I have  to  do  is 
verify  in  fact  the  removed  module  is  bad  and  then 
I'm  only  halfway  there.  I also  have  to  verify  that 
the  fault  in  the  module  is  such  that  it  was  not 
induced  by  something  else  in  the  black  box,  or  I 
have  not  done  proper  fault  isolation.  Otherwise 
I run  the  risk  of  having  a hidden  problem  in  the 
box  that  will  bite  me  later. 

Another  technique  which  got  introduced  into  the 
Titan  family  back  at  that  point  in  time,  and  which 
we  have  used  on  all  programs  since,  is  the  unveri- 
fied failure  discipline.  We,  in  fact,  did  fly  two 
pieces  of  hardware  that  had  unverified  failures  on 
Viking,  but  I'll  tell  you,  there  were  a lot  of 
agonizing  meetings  between  Jim  Martin,  who  was  the 
NASA  Program  Director,  and  myself,  before  we  bought 
off  on  them.  They  were  specific  instruments,  so 
that  they  could't  be  mission  catastrophic,  but  we 
could  have  lost  an  instrument.  We  had  a system 
that  allowed  us  to  know  where  the  risks  specifi- 
cally were.  If  you're  dealing  with  any  kind  of  a 
high  reliability  system,  you  had  better  have  that 
kind  of  discipline  where  the  project  manager  or 
technical  director  or  somebody  at  the  top  level  of 
the  project,  personally  assesses  the  risk  and  buys 
off  on  the  risk,  or  you're  not  managing  your  risks. 
Your  program  is  drifting  along,  essentially  out  of 
control,  from  a mission  success  point  of  view. 

The  next  on  I’ll  call  the  digital  myth.  1 
noticed  this  myth  a few  years  ago  as  digital  sys- 
tems became  more  prevalent  in  the  aerospace  world. 
During  the  Viking  days,  one  of  our  subcontractors, 
who  will  go  unmentioned,  was  producing  digital  air- 
borne equipment.  Most  of  the  people  in  his  organi- 
zation had  an  Ingrained  attitude,  "Well,  it's 


digital  equipment  and  therefore  you  should  expect 
ghosts  or  funnies,  to  happen  from  time  to  time." 

One  day  I was  walking  through  the  shop  where  we 
were  testing  one  of  our  first  pieces  of  flight 
hardware  add  I was  watching  the  guy  running  the 
system-level  test  on  this  box.  I wasn't  specifi- 
cally familiar  with  the  details  of  the  test,  but 
I could  sort  of  see  generally  what  was  going  on 
and  something  appeared  funny  and  he  stopped  and 
flipped  the  switch.  It  looked  to  me  like  it  was 
a repeat,  so  I asked  him.  He  said,  "Oh  yeah,  that 
was  a glitch,  it  appears  once  in  a while."  I 
said,  "What  do  you  do?"  "Well,  I reinitialize." 

No  documentation!  This  is  a modern-day  contractor 
in  our  business,  and  it  isn't  just  true  of  that 
particular  contractor.  The  feeling  was  --  well, 
digital  equipment  you  know,  has  those  kinds  of 
things.  It's  susceptible  to  it.  That's  baloney! 
That  just  says  you  don't  know  your  margins  and  you 
haven't  designed  the  system  properly.  When  we  got 
into  the  nitty-gritty  of  the  problem,  we  found  we 
had  test  equipment  in  that  shop  that  was  putting 
out  so  damn  much  noise,  we  didn't  know  what  we 
were  testing.  When  we  got  through  spending  about 
three  months  cleaning  up  the  test  equipment, 
because  it  was  a tough  job,  then  we  found  out  we 
had  some  design  problems  in  the  equipment  because 
of  the  pulse  rate  and  shape  and  what  have  you. 

We  really  had  lots  of  little  design  areas  in  the 
packaging  where  there  were  little  RF  generators 
that  we  had  to  eliminate.  After  we  got  all  done 
cleaning  it  up,  we  never  had  any  more  glitches. 

It  was  really  bloody  and  I had  to  go  all  the  way 
to  the  general  manager  of  the  facility  and  he  was 
the  only  man  in  the  facility  that  could  understand 
my  language  and  he  had  to  go  down  and  educate  his 
people,  all  the  way  down  through  the  line,  because 
they  had  spent  so  many  years  with  this  myth  that 
digital  equipment  always  had  glitches  from  time  to 
time  that  were  unexplainable  and  you  had  to  live 
with  it.  You  shouldn't  live  with  it  because  they 
are  dangerous. 

The  last  one  I want  to  talk  about  is  what  I'll  call 
the  software  change  myth.  I don't  know  how  much 
you're  going  to  get  into  software  here  in  your 
workshops  or  talks,  but  software  certainly  from  a 
mission  success  point  of  view,  ought  to  have  equal 
emphasis  to  hardware.  I don't  think  it  gets 
enough.  We  are  beginning  to  pay  a lot  of  attention 
to  it  in  our  company.  There  is  a prevalent 
attitude  and  there  has  been  for  a number  of  years, 
that  when  you  begin  to  get  into  the  inevitable  pro- 
blems in  system  or  spacecraft  development,  you  can 
fix  them  easiest  with  software.  You  get  your 
bright  young  guy  that  can  code  a piece  of  software 
and  he  says,  "I  got  the  invention  to  cure  your 
problem.  I'll  just  put  a jumper  in  and  put  a patch 
in  and  do  this,  that  and  the  other  for  you,  and 
the  problem's  fixed."  And  oh,  he  can  do  that.  He 
can  do  that  almost  over  night.  He  brings  his  little 
deck  of  cards  and  he  runs  it  and  provides  a new 
tape  and  gee,  you're  happy  because  the  machine 
works.  But  you  have  just  bought  a piece  of  the 
farm  potentially.  You  have  Invalidated  your  soft- 
ware, most  probably.  We've  all  been  guilty  of 
doing  that  to  some  degree  In  the  past.  We  really 
have  to  understand  the  total  implication  of  a soft- 
ware change  including  its  retest  and  revaluation. 
The  cost  of  that  software  change  is  often  as  much 
as  the  hardware  change,  because  you  have  an  exten- 
sive test  program  you  should  rerun,  to  have  the 
same  degree  of  assurance  that  you  had  before  you 
made  the  change.  You  tend  to  slide  by  that  fact 
time  after  time  because  you're  in  such  a box.  The 
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guy  says,  "Goa,  if  I got  to  change  the  hardware,  I 
got  to  go  back  and  order  new  PC  boards  or  some 
other  traumatic  thing."  It's  so  easy  to  create 
that  deck  of  cards  and  forget  the  validation  that 
you  really  should  go  through  to  provide  the  same 
assurance. 

I guess  my  half  hour’s  about  up.  I selected  those 
particular  items.  I could  go  on  for  many  hours. 

I selected  those  particular  items  because  they  were 
the  things  that  came  bubbling  into  my  mind,  fore- 
most, that  are  the  traps  that  I think  we  all  have 
to  pay  more  attention  to  stamping  out.  I just 
said,  we've  come  a long  ways  from  those  early  days 
when  we  really  expected  a lot  of  failures.  We  have 
a lot  of  people  in  this  romm  that  are  producing 
very  high  quality  hardware  now,  that  is,  in  fact, 
performing  in  space  for  years,  and  is  expected  to 
perform  years  more. 

In  my  tours  around  the  industry,  I observe  there 
are  still  these  kind  of  traps.  We  have  to  get  down 
into  the  bowels  of  our  organization  and  see  if 
we're  really  performing  some  of  the  things  the  way 
we  say  or  the  way  we  think  we're  doing.  We've  got 
to  improve  on  the  quality  of  our  implementation. 
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The  purposes  of  this  panel  are: 

1.  Present  to  you  summary  views  of  various 
aspects  of  piece  parts  management  for 
mission  assurance  from  various  perspec- 
tives: i.e.,  SAMSO,  NASA,  DESC,  and 
industry. 

2.  Stimulate  discussion  that  will  identify 
and  clarify  the  issues  involved  in  piece 
parts  management  for  assurance  of  space 
mission  success. 

The  Challenge 

There  is  a challenge,  common  to  all  these  space 
missions  and  systems:  the  challenge  to  assure 
mission  success. 

Speaking  in  terms  of  piece  parts,  assuring  success 
of  space  missions  is  indeed  a complex  and  demanding 
challenge. 

SAMSO  space  systems  and  missions  are  continually 
increasing  in  complexity.  Successful  on-orbit 
operations  for  SAMSO  systems  are  now  required  to 
live  without  failure  for  as  much  as  10  years.  In 
a system  with  up  to  70,000  piece  parts  and  no  on- 
orbit  maintenance  possible,  achieving  10  years  of 
reliability  is  indeed  a challenge! 

Secondly,  space  satellite  failures  can  be  caused 
by  any  one  of  the  thousands  of  electronic  parts 
that  comprise  an  on-orbit  system.  Redundancy  can 
help  reduce  failure  risk,  but  a common  failure 
mechanism  in  the  electronic  parts  of  redundant 
units  can  still  cause  system  failure.  Every  part 
of  a long  life  space  system  must  be  of  the  highest 
quality  to  prevent  premature  failure. 

Thirdly,  small  quantity  buys,  long  lead  delivery 
schedules,  market  influence  fluctuations,  program 
peculiar  spec,  requirements,  costly  qualification 
programs  and  lack  of  available  space  quality  parts 
have  caused  many  procurement  and  schedule  problems 
in  our  programs. 

And  after  we  had  received  the  parts,  SAMSO  has 
experienced  several  costly  flight  and  ground 
failures  attributed  to  faulty  electronic  parts. 

For  example,  one  of  our  major  space  communications 
programs  was  affected  by  failure  of  a launch 
vehicle.  We  lost  2 satellites  at  a cost  of 
S57  million  to  replace,  and  a significant  lapse  in 
DoD  space  communications.  This  failure  was  caused 
by  a particle  short  in  a single  transistor. 

Total  Parts  Management 

Meeting  this  challenge  of  space  mission  assurance 
requires  nothing  short  of  total  management  of  these 
critical  and  numerous  piece  parts. 

Since  any  single  part  can  cause  mission  failure, 
every  piece  part  (every  type  and  every  part  and 


lot  of  parts)  must  be  managed  in  a way  that  will 
increase  reliability  and  reduce  the  risk  of 
failure. 

All  piece  parts  must  be  managed  throughout  their 
life  cycle  if  space  quality  is  to  be  assured. 

Parts  management  and  reliability  assurance  must 
start  with  the  design  and  selection  of  parts  used 
in  a system  and  continue  throughout  the  applica- 
tion, manufacture,  procurement,  and  operation  of 
these  parts.  Mismanagement  at  any  step  can  cause 
mission  failure. 

It  is  very  easy  in  the  very  complex,  microscopic 
business  of  piece  parts  to  lock  onto  one  issue  or 
another  as  the  key.  Granted,  that  is  the  way  to 
address  and  solve  that  issue.  However,  space 
mission  success  requires  total  and  continual  man- 
agement of  all  of  the  aspects  and  issues  involved 
in  achieving  space  quality  and  reliability  in  all 
system  piece  parts. 

New  SAMSO  Parts  Policies 

For  many  years,  SAMSO' s policy  on  piece  parts  has 
been  to  require  (1)  use  of  the  highest  reliability 
parts  obtainable  and  (2)  management  of  part  selec- 
tion via  a part  control  program/board. 

During  the  past  year,  SAMSO  has  searched  for  other 
techniques  for  assuring  space  quality  parts  are 
used  in  all  space  missions. 

The  3 new  policies  shown  here  have  been  adopted  by 
SAMSO  and  approved  by  the  SAMSO  commander  for 
implementation  on  SAMSO  programs. 

The  class  "S"  parts  program  is  a joint  SAMSO,  NASA, 
DESC,  RADC,  industry  effort  to  create  military 
standard,  certified,  qualified,  available  space 
quality  microcircuits  and  semiconductors  for  use  in 
SAMSO  and  NASA  missions.  Other  panel  members  will 
discuss  the  class  "S"  program  in  more  detail. 

Coordinated  procurement  by  each  program  prime 
contractor  is  expected  to  reduce  part  procurement 
difficulties,  improve  standardization,  reduce  small 
quantity  buys,  and  improve  subcontractor  parts 
procurement  management.  Several  programs  (e.g., 
,AFSP  and  NASA/Viking)  have  successfully  implemented 
oordinated  procurements. 

Monitoring  of  the  part  manufacturer's  processing 
and  testing  of  program  parts  has  proven  successful 
on  several  programs.  SAMSO  intends  to  implement 
line  monitoring  by  the  program  prime  contractor  to 
assure  that  the  parts  delivered  for  space  missions 
are  truly  space  quality. 

Benefits 

Use  of  class  "S"  parts  and  line  monitoring  will 
result  in  higher  reliability  parts  in  space  and 
launch  vehicle  systems.  They  will  reduce  failures, 
schedule  delays,  rework  and  retest,  and  program 
costs. 

Class  "S"  parts,  available  in  the  military  QPL  will 
reduce  procurement  difficulties  and  eliminate  the 
need  for  costly,  duplicative  contractor  specifica- 
tion and  qualification  efforts. 
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Common  SAMSO/NASA/ Industry  class  "S"  requirements 
will  reduce  the  proliferation  of  program/company 
unique  specs  and  low  volume  unique  parts.  God 
knows  we  need  less  paperwork  and  less  legal  staff 
to  read  and  interpret  it. 

Joint  SAMSO/NASA  use  of  class  "S"  parts  and  imple- 
mentation of  spa^e  program  coordinated  procure- 
ments will  increase  the  procurement  volume  per  buy 
to  the  vendor,  increase  vendor  interest  in  class 
"S"  parts,  and  improve  both  the  reliability  and 
availability  of  space  quality  parts. 
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THE  IMPORTANCE  OF  ELECTRONIC  PARTS 
STANDARDIZATION  THROUGH  THE  MILITARY 
SPECIFICATION  SYSTEM 

Brigadier  General  George  K.  Patterson 
Commander,  DESC 

I welcome  this  opportunity  today  to  discuss  with 
you  the  national  significance  of  military  specifi- 
cations for  electronic  parts  standardization  to 
enhance  the  mission  assurance  of  space  programs. 

As  commander  of  an  organization  responsible  for 
integrated  management  of  one-sixth  of  all  items 
managed  by  the  U.S.  Government,  I am,  understand- 
ably, also  charged  with  the  task  of  managing  and 
conducting  a comprehensive  DoD  standardization 
program  for  electronic  parts,  used  in  a tremen- 
dously wide  variety  of  applications.  This  engine- 
ering standardization  mission  at  the  Defense  Elec- 
tronics Supply  Center  (or  DESC)  is  assigned  to  my 
directorate  of  engineering  standardization  and  is 
under  the  guidance  of  Colonel  Loren  Anderson.  To 
do  his  job,  "Andy"  supervises  the  actions  of  some 
200  engineering  personnel,  three-fourths  of  them 
electronic  engineers  and  electronic  technicians. 

The  roots  of  the  engineering  standardization 
mission  for  electronic  parts  at  DESC  run  deep  and 
date  back  to  the  Army  Signal  Corps  at  Fort 
Monmouth,  New  Jersey,  in  1943.  By  1949  the  organi- 
zation became  a tri-service  agency  and  was  better 
known  as  ASESA.  When  DESC  was  established  in  1962, 
the  functions  and  workloads  for  specifications, 
standards,  and  product  qualification  were  trans- 
ferred to  DESC  and  the  directorate  of  engineering 
standardization  formed. 

The  main  mission  of  my  engineering  directorate  is 
to  provide  standardization  support  services  to  the 
military  departments. 

One  of  our  most  important  jobs  is  to  provide  parts 
control  support  for  systems  program  managers.  In 
fiscal  year  1978  we  have  provided  parts  control 
assistance  to  284  design  contracts.  DESC  engineers 
have  provided  recommendations  on  the  selection  of 
40,000  electronic  part  types  and  on  the  adequacy  of 
7000  drawings.  The  life  cycle  cost  avoidance 
achieved  *rom  our  efforts  this  year  is  in  excess  of 
SI  10  million. 

The  backbone  of  the  electronic  parts  control 
program  at  0ESC  is  built  upon  our  mission  as  an 
agent  for  the  military  departments  in  the  prepara- 
tion of  military  specifications  and  military 
standards  and  the  administration  of  qualified  pro- 
ducts lists  (QPLs).  In  the  electronic  part  supply 
classes,  DESC  engineers  serve  as  an  agent  for  the 
military  departments  on  more  than  90*  of  the  specs 
and  standards  projects  established.  In  a given 
year  we  issue  more  than  1,200  standardization 
documents  impacting  an  estimated  50,000  parts.  In 
the  qualifications  area,  we  administer  QPLs  for 
150  general  military  specifications  for  electronic 
parts.  In  a typical  year,  we  audit  more  than  200 
facilities  and  evaluate  over  1,000  test  reports 
covering  thousands  of  product  types. 

DESC's  technical  capability  and  involvement  in  the 
development  of  specifications  led  the  DoD  to  desig- 
nate us  as  the  DoD  standardization  manager  for 
fourteen  federal  supply  classes,  covering  parts  in 
the  dynamic  electronic  part  technologies. 


As  you  see  from  this  diagram,  DESC  is  not  working 
in  a vacuum.  As  the  DoD  standardization  manager, 
we  must  work  closely  with  engineers  in  government 
and  industry  in  developing  an  effective  parts 
standardization  program.  Based  upon  the  needs  and 
inputs  troin  the  organizations  shown  here,  we  write 
more  military  specifications  and  standards  for 
electronic  parts  than  all  other  organizations 
combined  in  the  DoD. 

In  the  Department  of  Defense,  standardization  is 
the  law  and  not  a luxury.  Under  title  10  of  the 
U.S.  code,  the  Secretary  of  Defense  has  been 
directed  to  standardize  parts  to  the  highest  degree 
practicable.  A few  of  the  objectives  cited  in 
Section  2451  of  the  code  are  quite  appropriate  to 
my  message  today. 

In  managing  an  effective  standardization  program 
we  are  required  to  standardize  complex  electronic 
items  by  developing  and  using  single  specifications 
and  eliminating  overlapping  and  duplicate  documents 
describing  parts,  materials  and  processes  that  are 
generally  similar. 

While  developing  a single  set  of  specifications  we 
must  also  make  efficient  use  of  the  existing 
services  and  facilities  for  inspecting,  testing  and 
accepting  items  procured  to  military  requirements. 


MANAGEMENT  OF  THE  DoD  STANDARDIZATION  PROGRAM 
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assignments  in  the  major  electronic  parts  cate- 
gories has  been  delegated  down  to  me  as  the  DESC 
commander  and  head  of  the  DoD  standardization 
assignee  activity  for  the  fourteen  supply  classes 
shown  here.  You  will  note  that  the  kinds  of  elec- 
tronic parts  listed  are  those  same  products  in 
which  reliability  has  become  the  vital  link  to 
mission  assurance  in  space. 

Since  1972,  the  National  Aeronautics  and  Space 
Administration  (NASA)  and  the  Department  of 
Defense  have  been  working  together  on  the  develop- 
ment of  specifications  and  standards  for  transis- 
tors, diodes,  and  microelectronic  circuit  devices. 
In  February  1974,  the  DoD-NASA  memorandum  of  under- 
standing was  updated  to  include  resistors  and 
capacitors.  Under  the  memorandum,  it  was  agreed 
that  NASA  would  be  accorded  all  the  standardization 
authority  and  responsibility  comparable  to  that  of 
the  Army,  Navy  and  Air  Force.  As  you  can  see, 

NASA  is  now  a very  important  facet  in  four  of  the 
fourteen  FSCs  delegated  to  me  for  standardization 
mangement. 

Standardization  has  long  been  an  important  element 
of  military  logistics,  design,  research  and  engine- 
ering. From  a national  standpoint,  it  has  often 
provided  the  essential  ingredient  for  profitable 
production  and  distribution  of  goods  in  the 
civil ian  market. 

In  the  electronic  parts  area,  standardization 
represents  a 35  year  evolution  of  effort  practiced 
by  the  military  services  in  establishing  contract 
requirements. 

However,  lessons  learned  from  World  War  II,  Korea, 
South  East  Asia  and  the  intervening  state  of  inter- 
national tension,  visibly  demonstrate  a national 
need  for  management  attention  to  the  standardiza- 
tion of  highly  reliable  parts  for  space,  missile 
and  avionic  applications. 

In  today's  world  the  problems  driving  the  need  for 
standardization  through  the  military  specification 
system  are  easily  apparent: 

1.  While  electronic  equipment  is  extremely 
complex  and  costly,  the  available  real 
dollars  in  DoD  and  NASA  to  buy  such  hard- 
ware is  diminishing. 

2.  When  we  can  afford  to  buy,  we  find  the 
cost  to  own  the  hardware  is  often  double 
the  original  investment,  when  life  cycle 
costs  are  considered.  Each  time  a new 
weapon  system  enters  the  PoD  inventory 
it  brings  with  it  thousands  of  new  items 
of  repair  parts,  spares  and  support 
equipment.  Depending  upon  the  complex- 
ity, uniqueness  and  end  item  application 
of  a particular  device  type,  the  cost 
impact  to  document  and  test  could  range 
from  $2,000  to  $25,000  for  the  non- 
standard part  type  applied  in  a new 
design.  Considering  the  quantity  of 
nonstandard  parts  used  in  a typical  dev- 
elopment program,  the  engineering  and 
testing  cost  for  nonstandard  items  is 
becoming  prohibitive. 

3.  Manufacturers  of  equipment  and  systems 
are  frequently  faced  with  the  fact  that 


parts  and  components  are  not  available 
to  meet  the  ever  increasing  demand  for 
better  reliability,  maintainability, 
transportability,  and  performance.  The 
net  result  is  a great  volume  of  engineer- 
ing talent  and  money  expended  in  search- 
ing for  improvdd  parts  and  inspecting  and 
testing  such  parts  to  verify  and  assure 
rel iabi 1 ity. 


4.  The  independent  solution  by  each 

contractor  of  the  problem  is  often  the 
development  of  a new  nonstandard  part. 
These  independent  solutions  to  the  same 
problem  result  in  a steady  increase  in 
the  number  of  varieties  of  items  and  a 
wasteful  duplicative  effort  in  develop- 
ment, testing  and  inspection.  Resources 
will  continue  to  be  diverted  until  all 
our  designers,  engineers  and  logistic 
managers  have  and  use  up-to-date  specifi- 
cations, standards  and  QPLs  which  contain 
solutions  to  these  repetitive  problems. 

The  improper  use  of  specifications  and  standards 
are  a cause  of  much  concern.  The  Office  of  Manage- 
ment and  budget  and  the  Under  Secretary  of  Defense 
for  Research  and  Engineering  are  now  taking  a 
detailed  look  at  the  origination,  maintenance  and 
application  of  specifications  and  standards.  Both 
Congress  and  the  General  Accounting  Office  have 
condemned  specs  and  standards  as  sources  of  poor 
performance,  goldplating,  excessive  delays,  and 
unnecessary  costs.  In  July  1974,  at  the  request 
of  DoD,  the  Defense  Science  Board  created  a task 
force  on  specifications  and  standards  with  a pri- 
mary objective  of  developing  recommendations  for 
improving  the  use  of  these  documents. 

In  April  1977,  the  report  of  the  task  force  on 
specifications  and  standards  (or  SHEA  Report) 
identified  many  documents  containing  procedure 
techniques  considered  to  be  a major  factor  in 
driving  costs  when  misused  or  not  tailored  to  the 
specific  needs  of  a program.  Most  of  the  documents 
identified  as  cost  drivers  in  the  SHEA  Report  are 
the  "ill ity"  documents  used  for  general  design 
requirements,  environmental  requirements,  test 
methods,  reliability,  maintainability,  configura- 
tion control,  human  factors,  safety  and  other 
similar  documentation. 

The  SHEA  Report  recognized  the  need  for  assuring 
that  essential  operational  capability  requirements 
were  not  sacrificed  when  tailoring  requirements  to 
support  a particular  system.  On  the  other  hand, 
the  report  was  quite  clear  in  its  claim  that  DoD 
was  over-applying  and  mis-using  its  specifications 
and  standards. 

We  were  pleased  with  the  comment  in  the  SHEA 
Report  concerning  the  progress  made  by  DESC  in  the 
parts  control  program  and  in  managing  standardiza- 
tion effort  for  electronics  parts.  The  joint 
effort  of  DESC,  the  military  departments,  NASA, 
other  government  agencies  and  trade  associations 
has  made  military  specifications  and  standards  the 
"National  Corporate  Memory"  for  electronic  parts 
procurement  and  standardization.  We  now  have  the 
opportunity  to  apply  this  "National  Corporate 
Memory"  to  electronic  parts  used  in  the  space 
mission  environment. 
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Recently  it  was  my  pleasure  to  sign  a joint  agree- 
ment between  SAMSO,  NASA,  and  DESC  on  operating 
procedures  governing  the  administration  and  tech- 
nical aspects  of  microcircuit  certification 
requirements  of  MIL-M-38510D  add  MIl-STD-976. 

I believe  we  have  achieved  a significant  milestone 
in  the  evolution  of  Class  S parts  under  the 
umbrella  of  MIL-M-38510D.  More  significantly,  the 
joining  of  DESC,  NASA  and  SAMSO  to  invoke  the 
Class  S qualification  and  certification  program  is 
a money  winner  all  the  way. 

In  the  past,  we  three  have  been  essentially 
operating  on  our  own  to  qualify  high  reliability 
parts.  DESC  has  administered  qualification  for 
Class  B and  C devices,  and  has  authorized  testing 
and  evaluated  test  reports  for  the  old  Class  A 
devices.  NASA  has  conducted  their  own  line 
certification  audits  for  the  old  Class  A devices. 
SAMSO  has  been  administering  their  own  quality 
program  under  MIL-M-38510B.  The  net  result  has 
been  duplication  and  overlap  of  audits  at  the 
manufacturers'  plants  by  the  three  activities.  In 
addition  to  audit  overlap,  audit  uniformity  and 
consistency  have  suffered.  Now  that  the  new 
Class  S had  been  placed  under  MIL-M-38S10D,  super- 
seding M1L-M-38510B,  and  MIL-STD-976  has  been 
issued,  a new  look  has  been  taken  at  the  qualifi- 
cation and  certification  requirements  of  micro- 
circuits. 

I would  like  to  explain  how  DESC,  SAMSO,  and  NASA 
will  work  together  in  the  Class  S qualification 
program.  The  Defense  Electronics  Supply  Center, 
DESC-EQ,  as  the  DoD  focal  point  for  the  JAN 
microcircuit  program,  will  perform  all  administra- 
tive work  on  Class  S devices.  This  includes  paper- 
work involved  in  the  qualification  of  microcircuits 
and  the  certification  of  manufacturers'  facilities. 
DESC,  NASA,  and  SAMSO  will  agree  on  a chairman  for 
an  audit.  For  potential  Class  S line  audits, 
chairmanship  will  normally  be  either  NASA  or 
SAMSO.  Following  an  audit,  DESC  will  issue  audit 
deficiency  letters,  coordinate  corrective  actions, 
issue  certification  letters,  authorize  testing, 
evaluate  test  reports,  and  issue  notification  of 
qualification.  For  Class  "S"  devices  NASA  and 
SAMSO  will  be  voting  members  on  the  technical 
aspects  of  DESC  administrative  actions. 

The  key  point  in  Class  S certification  is  the 
provision  for  a joint  team  audit  composed  of 
individual  members  from  each  government  activity 
that  has  an  interest.  This  will  result  in  several 
advantages: 

1.  Minimize  company's  time  spent  during 
audits 

2.  Eliminate  audit  duplication  and  overlap 

3.  Reduce  overall  audit  expenses 

4.  Improve  uniformity  of  audits 

To  effect  this  last  advantage,  we  plan  to  restrict 
the  number  of  participants  performing  the  audit  to 
provide  an  efficient  team.  Only  the  people  essen- 
tial for  the  audit  will  be  in  attendance.  This 
will  ensure  cost  effectiveness  and  provide  a 
uniform  audit  in  minimum  time. 


A significant  ingredient  for  a successful 
qualification  program  is  fair  and  equitable  treat- 
ment of  all  manufacturers.  We  need  to  be  on  guard 
that  the  requirements  which  we  set  forth  are  in 
fact  needed  and  realistic  and  that  they  will 
satisfy  our  needs.  Fair  and  equitable  treatment 
will  be  improved  through  the  joint  efforts  of  DESC, 
NASA,  and  SAMSO.  Another  significant  ingredient 
for  a successful  qualification  program  is  a market 
for  devices.  The  market  place  for  Class  S devices 
should  now  be  enhanced  through  the  combined  usage 
of  both  NASA  and  SAMSO. 

Aside  from  the  qualification  program,  other  goals 
need  to  be  accomplished.  With  the  incorporation  of 
class  S devices  in  MIL-M-38510D,  we  plan  to  strive 
to  establish  Class  S devices  in  other  electronic 
part  categories  as  well. 

Let  us  take  a look  at  future  Class  S activity  in 
other  mil  specs. 

In  December  1977,  DESC  as  agent  for  the  Naval 
Electronic  Systems  Command  Preparing  activity, 
revised  MIL-S-1 9500,  the  general  specification  for 
semiconductors.  By  Revision  F to  MIL-S-1 9500 , 

Class  "S"  space  parts  requirements  were  placed  in 
the  document.  The  JAN  Class  "S"  is  the  fourth  and 
highest  level  of  product  assurance  in  MIL-S-1 9500 
and  is  preferred  in  space  equipment  over  the  lower 
assurance  levels  such  as  JAN,  JANTX,  and  JANTXV. 
Audits  of  JAN  S certified  lines  are  required  by 
Section  3 of  the  specification.  The  audit  team 
procedure  will  follow  the  same  procedure  as  used 
in  MIL-M-38510. 

We  are  now  taking  a look  at  some  of  the  other  mil 
specs  that  may  be  adaptable  to  the  Class  S concept. 
Established  reliability  or  "ER"  specifications  for 
resistors  and  capacitors  are  now  under  considera- 
tion. We  are  also  examining  other  mil  specs  cover- 
ing product  areas  such  as  EMI  filters,  switches, 
relays,  connectors,  crystals  and  RF  coils. 

We  are  optimistic  that  the  mil  spec  system  for 
electronic  piece  parts  can  be  adapted  to  the  com- 
plex mission  assurance  needs  of  space  and  missile 
programs.  I believe  it  will  be  a major  milestone 
in  the  achievement  of  our  expectations  in  the 
awakening  of  a new  mil  spec  standardization  system 
in  the  best  interests  of  the  nation. 


19 


H 


...... — 


GENERAL  OBSERVATIONS  ON  PARTS  MANAGEMENT  AND 
A REVIEW  OF  THE  GPS  EXPERIENCE 

Martin  L.  Adams 

Manager,  Systems  Effectiveness 
Rockwell  International 


Observations 


4.  Overloaded  the  screening  agency  and  had 
to  inject  massive  management  and  techni 
cal  effort  to  make  deliveries. 

5.  Overall  - excellent  quality  parts  were 
achieved  but  cost  was  too  high  and 
schedule  risks  were  unacceptable.  A 
better  technique  was  needed. 


1.  On  GPS  EEE  parts  are  98  percent  of  the 
functional  elements  of  the  Satellite- 
Huge  opportunity  for  management  techni- 
ques to  influence  reliability  of  S/C 
thru  the  parts  program. 

2.  Critical  Areas  to  be  Aware  of 


On  the  follow-on  2 vehicles  we  had  an  opportunity 
to  pioneer  a better  approach. 

Better  Approach 

1.  A pooled  procurement  - emphasis  on  using 
the  original  mfr.  to  do  the  rescreening, 
with  less  reliance  on  screening  facility 


a.  Understand  Requirements 

Management  may  not  fully  understand 
the  technical  requirements,  and 
underestimate  the  cost  and  staff 
requirements. 

e.g..  It  may  not  be  clear  that  the 
JAN  and  ER  devices  require 
rescreening. 


After  agreement  with  each  subcontractor 
on  parts  to  be  pooled,  quantity,  need 
date,  RI  prepared  specifications  and 
negotiated  with  manufacturers  for  best 
price  and  delivery. 

RI  placed  master  P.0,  and  subcontractors 
issued  independent  P.O.'s  referencing 
the  master  P.0. 


b.  lead  Times 


Ambitious  vehicle  schedules  may 
ignore  the  real  parts  lead  times. 
SAMSO  has  to  lead  here,  because  no 
bidder  wants  to  degrade  his  competi- 
tiveness by  realistically  assessing 
that  the  parts  program  lead  times 
may  not  support  the  hardware  needs. 
Later,  Mission  Assurance  may  be 
degraded  because  of  compromises  in 
part  pedigree  to  meet  schedules. 

c.  Parts  Not  Available 

Know  in  advance  that  except  for 
Class  S microcircuits,  parts  meeting 
SAMSO  requirements  are  not  procurable 
off-the-shelf.  Also,  because  of  low 
volume  and  special  processing  con- 
trols, manufacturers  may  not  be 
cooperative. 

There  are  some  techniques  to  over- 
come mfr.  resistance  which  we  have 
successfully  deployed  and  will 
discuss  later. 

After  framing  the  above  issues.  - then  discuss  the 

GPS  experience  - 

On  the  first  6 systems  - 

1.  Took  careful  effort  to  assure  all  sub- 
contractors understood  requirements. 

a.  Visits  to  each 

b.  Overall  conference 

2.  Quickly  assessed  that  the  small  suppliers 
and  low  volume  users  needed  help. 

3.  Developed  a pooled  procurement  concen- 
trating on  using  screening  facility  to 
upgrade  parts. 


4.  RI  provided  technical  management,  source 
inspection,  status  reporting  and  per- 
formed most  of  the  DPA's. 


5.  Manufacturer  delivered  parts  and  data 
directly  to  subcontractor. 


Advantages 
1. 


Risks  spread  over  20  manufacturers  and 
one  screening  facility. 


2.  Most  schedules  met  or  exceeded. 

3.  Manufacturer  put  in  extra  effort  because 
of  size  and  $ value  of  order  and  aware- 
ness of  requirements  that  must  be  met  for 
lot  acceptance. 

4.  Substantial  cost  savings  due  to  larger 
quantities,  elimination  of  multiple  lot 
charges  (DPA's,  SEM,  lot  minim's)  and 
central ization  of  PQA,  technical  and 
administrative  support. 

Challenges  for  SAMSO 

1.  Take  the  lead  in  making  the  "S"  type 
parts  available  for  more  circuit  types 
and  styles. 

2.  Encourage  schedule  realism  in  proposals. 
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GENERAL  OBSERVATIONS  ON  PARTS  MANAGEMENT 
AND  THE  DSCS  III  PROGRAM  EXPERIENCE 

Paul  Dick 

Manager,  Product  Assurance,  DSCS  III 
General  Electric  Company 


THREE  KEY  ELEMENTS  OF 
NEW  SAMSO  PARTS  POLICIES 


1 - CLASS  "S"  PARTS  USAGE 

2 - COORDINATED  PROCUREMENT 

3 - PART  MANUFACTURER  MONITORING 


GENERAL  ELECTRIC'S  APPLICATION  OF 
THESE  THREE  KEY  CONCEPTS  TO 
THE  DSCS  HI  PROGRAM 


GENERAL  ELECTRIC'S  APPLICATION  OF 
THESE  THREE  KEY  CONCEPTS  TO 
THE  DSCS  IH  PROGRAM 


1 - HIGHEST  RELIABILITY  PART  USAGE 


2 - PROGRAM  INTEGRATED  PARTS  SYSTEM  (PIPS) 


3 - PARTS  QUALITY  MONITORING  ACTIVITIES 


1 - HIGHEST  RELIABILITY  PART  USAGE 

— 

2 - PROGRAM  INTEGRATED  PARTS  SYSTEM  IPIPSI 

3 - PARTS  QUALITY  MONITORING  ACTIVITIES 

2 - DSCS  III  PROGRAM  INTEGRATED  PARTS  SYSTEM  IPIPS) 


• PRIOR  EXPERIENCE  BACKGROUND  • PROCUREMENT  PHASE 

• PIPS  PROGRAM  SCOPE 

• DEFINITION  PHASE 


PRODUCTION  AND  ACCEPTANCE  PHASE 

f HAITI  R 

V.FRSCHRUMH1 


1 - HIGHEST  RELIABILITY  PARTS  USAGE 


PARTS  PROGRAM  CONTROL  DOCUMENTATION 

PROGRAM  PtAN  SUPPORT  PROCEDURES  SPECIFICATIONS 

USC  IS  MAO*  Of  n TYPES  Of  THE  ORIGINAL  CLASS  S"  TTL  LOIN  POWER 
MICROCIRCUITS  ON  MR.  MMNS 

OTHER  ACTIVE  DEVICES  I SEMICONDUCTORS  AND  1C  SI  AND  UNIQUE 
PASSIVES  ARE  PROCURED  TO  TECHNICALLY  EQUIVALENT  S CLASS 
REQUIREMENT  SPECIFICATIONS  PREPARED  BY  QE 
COMMON  TYPE  PASSIVES  ARE  PROCURED  TO  HIGHEST  RELIABILITY 
LEVEL  POSSIBLE  CONSISTENT  WITH  UPGRADED  ER  MIL  REQUIREMENTS 
SPECIAL  DEVICES  GIVEN  ADDITIONAL  STRONG  CONTROLS 
- LSI  t - OaAPFCT  • CMOS  HYBRID*  - CHIPS 


PIPS  PROGRAM  SCOPE 

USER  IDENTIFIES  PARTS  NEEDS  TO  PRIME  CONTRACTOR 

PRIME  CONTRACTOR  CONSOLIDATES.  STANDARDIZES  AND  INTEGRATES 

REQUIREMENTS  WITH  USERS 

PRIME  CONTRACTOR  SPECIFIES.  ORDERS.  EXPEDITES  AND  DELIVERS 
ALL  COMMON  USAQE  PARTS  FOR  11  SUBCONTRACTORS  MAKING  IS 
DIFFERENT  PROOUCTS.  IN  ADDITION  TO  PARTS  NEEDED  BY  PRIME  FOR 
HIS  OCSIGNS 


MKOM  TRACT  00 


M •CONTRACTORS 


SIGNIFICANT  COST  REDUCTIONS 

- UNIT  COSTS  LEM 
FfWtA  LOT  CHANGES 

- ELIMINATION  OF  DUPLICATE  LANON  EFFORTS 
REDUCTION  OF  TRAVEL  ANO  LIVING  COSTS 

UNIFORM  QUALITY  PROGRAM 

COMMON  PROCESS  REVIEWS  AND  SURVEYS 
FFWER  SOURCES  OF  SUPPLY 
GREATER  TRACEABILITY  OF  DATA 
SCHEDULE  MAINTENANCE 
TIMELY  ORDERING 
PROGRAM  MANAGEMENT  VISIBILITY 

- PROBLEM  RESOLUTION  ANO  WORKAROUNDS 


DEFINITION  PHASE 

USER  REQUIREMENTS  REVIEW 
USERS  REQUIREMENTS  INTEGRATION 
SPECIFICATION  PREPARATION 
PMPCB  REVIEW  AND  COMMENTS 
COMMENTS  INCORPORATION 

PART  MANUFACTURER  - USER  - PMPCB 

SPECIFICATION  APPROVAL 
CHANGE  CONTROL 
PROGRAM  "ALERT  SYSTEM 


PRODUCTION  AND  ACCEPTANCE 
PHASE 


• GE  REVIEW  OF  SEM  ON  ACTIVES 

• 100%  PRECAP  VISUAL  ON  ACTIVES 
AND  DESIGNATED  PASSIVES  BY 
GE  ICSII  AND  GOVERNMENT  IGSII 

• PROCESS  REVIEWS  ANO  AUDITS  DURING  PRODUCTION 

• SCREENING  AND  BURN  IN  PERFORMED  TO  GE  APPROVED  SCREENING 
PLANS 

• EARLY  DPA  ANO  RADIATION  TEST  PROGRAMS 

• TEST  WITNESSING  ON  CRITICAL  OEVICES 

• PART  MANUFACTURERS  DATA  REVIEWS  ON  RECEIPT  OF  PARTS 

• USER  INSPECTION  AND  PARAMETER  TESTS  AND  DATA  REVIEWS 

• PRIME  CONTRACTOR  RESPONSIBLE  FOR  LOT  TEST  PROGRAMS 

DPA  RADIATION  LAT 


GENERAL  ELECTRIC'S  APPLICATION  OF 
THESE  THREE  KEY  CONCEPTS  TO 
THE  DSCS  III  PROGRAM 


1 - HIGHEST  RELIABILITY  PART  USAGE 


2 - PROGRAM  INTEGRATED  PARTS  SYSTEM  (PIPS) 


3 - PARTS  QUALITY  MONITORING  ACTIVITIES 


3 - PARTS  QUALITY  MONITORING  ACTIVITIES 


• PRE  AWARD  SURVEYS 


• PRECAP  VISUAL  INSPECTIONS 


LOT  ACCEPTANCE  TEST  PROGRAMS 


LOT  ACCEPTANCE  PROGRAMS 

SCANNING  ELECTRON  MICROSCOPE  (SEMI  EVALUATIONS  Of  WAFERS 

MmOAATEO  CIRCUIT  a 

MCTAUKAIION  OVER  0*101  STIRS  __ 


LOT  ASSURANCE  TESTS 

iomq  tirm  burr  in  stability 


RADIATION  SCREENING 
LOT  SAMRLf  run 
LATCH  UR  TESTS 


DESTRUCTIVE  PHYSICAL  ANALYSIS  IOPAI 


PERCENT  DEFECTIVE  ALLOWABLE  (POA) 
TREAT  WITH  CAUTION 


USER  RECEIVING  INSPECTIONS  ANO 
TESTS  OF  PARTS  - NOW  EFFECTIVE  IS 
THIS  POLICY? 


YEAR 

QUANTITY 

QUANTITY 

PROCESSED 

REACTED 

1111  (In  QTR) 

isa.ni 

Mil 

177.471 

is.m 

an,  ns 

as.au 

aa.iM 

II.M7 

1.B23.BB1 

aa.iai 

1I7J 

447441 

4S.S37 

an.ME 

HIM 

asi.iM 

11.111 

ait  im 

14.EM 

(I6HT  ANO 

ONC  OUARTER 
YEAR  FCRIOO 

J.MS.S14 

1IM71 

'£A  Vl  *Z  Dt 


OPA  CANDIDATES 


ACTIVES 

• DIODES  • 

• HYBRIDS  • 

PASSIVES 

• CERAMIC  CAPACITORS 

• CRYSTALS 


ICS 

TRANSISTORS 


• CERAMIC  CAPACITORS  • FILTERS 

• CRYSTALS  . REUAYS 

RESULTS  TO  DATE  (42?  ' OTS) 

• APPROXIMATELY  2%  OF  LOU  **f 
DISQUALIFIED  BASED  ON  OPA  RESw1 

• CURRENT  BAD  OUVS  ARE 

CERAMIC  CAPACITORS 
RELAYS 


CONCLUSIONS 

' PARTS  HERITAGE  IS  AN  IMPORTANT  SELECTION  FACTOR 

QUALIFIED  CONSISTENT  SOURCE  OOOO  USAGE  RECORD 
1 THE  BEST  TYPE  OF  PART  TO  USE  FOR  A LONG  LIFE  MISSION  IS  THE 
HIGHEST  RELIABILITY  AVAILABLE 

QUALITY  MUST  BE  DE8IGNE0  INTO  A PART.  IT  IS  NOT  A PRODUCT  OF 
SCREENING 

A WtU.  MANAGED  COORDINATED  PROCUREMENT  PROGRAM  OFFERS 
COST.  QUALITY  AND  SCHEDULE  ADVANTAGES 

PRECAP  VISUAL  INSPECTIONS  (CSII  ARE  A VERY  IMPORTANT  PRODUCT 
QUALITY  TOOLGATE  DURING  PRODUCTION 

DPA  IS  AN  EFFECTIVE  TOOL  TO  FIND  POTENTIALLY  MARGINAL  UNITS 
USER  RECEIVING  INSPECTIONS  AND  TESTS  ON  PARTS  IS  A VERY 
VALUABLE  FINAL  CHECKPOINT 

PARTS  MANAGEMENT  IS  TRULY  A’WOMB  TO  TOMBr'CYCLE 
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NASA  STANDARD  PARTS  PROGRAM 

Joseph  L.  Murphy 
Manager,  Parts  and  Material, 
Program  Assurance  Division 


Established  to  Deal  With: 

• Small  quantity  buys 

• Reliability  demands  vs.  high  cost 

• Multiple  specifications 

• Duplicative  engineering  efforts 

• Long  delivery  schedules 

• Market  influence  and  fluctuation 

• Failures  and  expensive  rework 

Purpose 

• Increase  availability  of  reliable 
microcircuits,  transistors,  diodes, 
capacitors  and  resistors 

• Reduce  overall  costs  --  eliminate 
duplication 


Applicability 


Scope 


In-house  and  contracted  work 


• New  programs  --  flight  hardware  and 
mission  essential 

• Ground  support  equipment 
NASA  Events 

Leading  to  Joint  Parts  Program 

1964  NPSC  Microelectronics  Subcommittee 

1972  Microelectronics  Reliability  Program 

NASA/DoD  Agreement 

1974  NASA  Std  Parts  Program 

1975-76  NASA/SAMSO  coordination  started  (space 
parts  working  group) 

1977  NASA/SAMSO  activities  accelerated 

1978  Joint  requirements  being  implemented 


• Incorporate  Class  "S"  into  mil  specs 

• Develop  Class  "S"  certification 
requirements 

• SAMSO/NASA/DESC  certifications 

• Eliminate  audit  duplication  and 
overlap 

• Reduce  audit  expense 

• Improve  uniformity  of  audits 

• Class  "S"  qualifications 

• SAMSO/NASA  preferred  parts  list 

• Procurement  volume 

Accompl ishments 


NASA-SAMSO  agreements  on  Class  S 
Revised  MIL-STD-975  (std  parts  list) 
Issued  MIL-STD-976  (certification) 

Certification  guidelines  issued  - DESC- 
EQE-44 

SAMSO/NASA/DESC  operating  procedure 
signed 

Conversion  of  Class  A to  S and  new  certi- 
fication started 

MIL-M- 38510  and  MIL-STD-883  revised/ 
issued 

MIL-S-1 9500  and  MIL-STD-750  revised/ 
issued 

Detail  specifications  initiated  - 38510, 
19500 

ER  specification  study  initiated 


NASA  Standard  Parts  List 

MIL-STD-975  - released  Jan  1976/revised  Aug  1977 
Covers  - microcircuits  - MIL-M- 38510 


Transistors 

Diodes 


MIL-S-1 9500 


Capacitors 

Resistors 


"ER"  Spec 
Includes  - two  quality  grades 


Joint  SAMSO/NASA  Parts  Approach 

• Develop  Class  "S"  (space  quality) 
requirements 

• Microcircuits 

• Diodes/transistors 

• Hybrids 

• Other  devices 


Quality  Grades 

Microcircuits 

Semiconductors 

Resistors 

Capacitors 


Grade  I 
Class  S 

JAN  S 

"S"  (0.001%/ 
1000  hrs) 


Grade  II 
Class  B 

JAN  TX 

"P"  (0.1%/ 
1000  hrs) 
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Application  of  Quality  Grades 


NASA 

Shuttle 

Instrument 

Classes 


Payload  Category 


Qual ity 
Grades 


Center  parts  specialists  can  devote 
more  time  and  attention  to  nonstand- 
ard parts 

Provides  standardized  baseline  for 
project  common  procurement  (larger 
volume) 


"A" 

Maximum 

Confidence 


Upper  Stage 

Planetary,  deep  space, 
geosynchronous 


Directly  deployed 
Earth  applications. 


1 and  2 


Benefits  of  Project  Parts  Consolidated  Procurement 
• Lower  unit  costs 


science 


Attached  experiments 

Scheduled  or  critical 
objectives 


1 and  2 


One  central  procurement  contract  - 
eliminates  redundant  cost  associated  with 
multi-tiered  procurements 


Shorter  delivery  times 


Full  technical  control  at  program  level 
of  parts  quality  and  reliability 


"B" 

Normal 

Confidence 


Upper  stage 
Directly  deployed 
Attached  experiments 
Space  Available 


and  2 


Leverage  of  high  volume  purchasing 


Total 
Range  of 
Available 


NASA-Wide  Consolidated  Procurement  Study  - 
Estimated  Results 


Parts 

20%  Increase  (net)  in  NASA  parts  inhouse  support 

12%  Decrease  in  parts  cost 

Activity  Expansion 

37%  Reduction  in  contractor  parts  engineering 

Hybrids 

Thermisters 

support 

Connectors 

Transformers 

49%  Reduction  in  contractor  parts  material 

F i 1 ters 

Crystals 

overhead,  G&A,  profit 

Fuses 

Inductors 

Relays 

Wire  and  Cable 

$35  million  savings  first  five  years  of  operation 

$150  million  savings  first  ten  years  of  operation 

NASA  Proqram/Project  Benefits 

• Supports  payload  policy 

Consolidated  Procurement  Experience 

• Easily  understood  baseline  for  design 

Viking  Program 

standardization  allows 

cost  estimates 

from  experience 

Two  quality  grades  of  parts  - more 
reliability  experience 

Lower  project  engineering  costs 


2.5  million  parts  - 90%  delivered  to  18  con- 
tractors in  less  than  a year 


Fewer  specif ications  to  prepare  and 
coordinate 


100,000  Class  A digital  microcircuits  - 90% 
of  order  and  100%  of  line  items  delivered  in 
10  months 


• Fewer  qualification  tests 
Lower  part  procurement  costs 


Larger  nationwide  usage  of  same 
parts  - joint  NASA,  SAMS0 
requirements 

Class  S microcircuits  15%  lower  than 
MSFC  preferred 


Suppliers  familiar  with  specifica- 
tions - larger  cost  amortization 
base 


Estimated  savings  $20  million 
HEA0-"A" 

Parts  valued  at  $1.1  million 
Estimated  savings  - 10%  to  20% 

Lead  time  shortened  by  4 to  6 months 
RFP  bid  cycle  was  shortened  by  30  days 


Parts  stocked  by  suppliers  or  shorter 
delivery 


Results  - HEA0  "A”  passed  8 month  performance 
period  on  April  12,  1978  without  any  known 
electrical  part  failure 
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Where  Do 

We  Go  From  Here 

• 

Promote  standard  parts  use 

• NASA  - Grades  1 and  2 

• SAMSO  - Class  S 

• 

Accelerate  Class  S qualifications 

• 

Continue  to  review  and  develop  detail 
specifications 

f 

• 

Develop  other  Class  S part  types 

• 

Promote  contractor  parts  consolidated 
procurement 

t 

Continue  and  expand  dialogue  and 
coordination 

• User  agencies 

• Manufacturers 

• System  contractors 

t 

Improved  packages,  materials,  processes 

DESIGN  PANEL 


Mr.  Paul  E.  Wright,  Moderator 
Vice  President,  Engineering 
Government  Systems  Division 
RCA  Corporation 


This  Panel  will  deal  with  the  subject  of  designing 
for  mission  assurance.  As  you  can  see  from  the 
list  of  names  shown  on  the  viewgraph,  we  are  for- 
tunate to  have  a distinguished  group  of  senior 
executives  from  the  Aerospace  Industry.  This  Panel 
brings  well  over  100  years  of  technical  and  manage- 
ment experience  to  the  subject.  The  majority  of 
successful  space  missions  in  this  country  since  the 
beginning  of  the  space  era  have,  in  one  way  or 
another,  been  touched  by  the  members  of  this  Panel. 
I am,  therefore,  pleased  and  honored  to  introduce 
them  to  you. 

As  the  block  diagram  indicates,  design  is  one  of 
many  steps  in  the  process  linking  requirements  with 
the  launch  of  a spacecraft.  Our  concern  is  with 
mission  assurance.  Mission  Assurance  clearly 
extends  beyond  the  date  of  launch;  indeed  it 
extends  to  the  end  of  satellite  life.  But  to  a 
first  approximation,  the  controllable  variables  in 
mission  assurance  are  those  shown  in  this  block 
diagram.  The  next  diagram  lists  a few  of  the  ele- 
ments of  designing,  as  that  word  is  used  in  the 
context  of  today's  meeting.  An  important  issue  for 
the  spacecraft  design  team  is  the  balance  of  per- 
formance against  protection.  In  other  words,  it  is 
the  trade-off  of  high  performance  against  an  insur- 
ance policy  for  success  of  the  spacecraft.  Another 
important  issue  that  will  be  dealt  with  by  the 
members  of  this  Panel  is  the  question  of  testing  - 
where  should  it  be  done,  how  extensively  should  it 
be  done  and  how  can  one  design  the  spacecraft  to 
facilitate  appropriate  and  comprehensive  testing. 
Clearly,  both  government  and  industry  share  in  the 
responsibility  for  designing  for  mission  assurance. 
The  generation  of  specifications  and  requirements 
by  the  Government  strongly  influence  the  degree  to 
which  industry  can  successfully  design  for  mission 
assurance.  Another  design  issue  is  the  matter  of 
hardware/software  tradeoff  because  of  the  rapid 
maturity  of  computers  implemented  in  LSI  and,  even 
more  pointedly,  because  of  the  advent  of  the  micro- 
processor and  the  ability  to  distribute  computa- 
tional horsepower  throughout  the  spacecraft.  Care- 
ful modeling  of  the  design  before  it  is  finalized 
is  also  important  to  mission  assurance. 

Our  most  important  overall  objective  in  spacecraft 
design,  given  the  achievement  of  a threshold  level 
of  performance,  is  to  achieve  high  quality  -quality 
that  will  insure  mission  success.  Pursuant  to  that 
goal,  traceability  of  the  design  is  a part  of 
achieving  that  level  of  quality.  It  is  important, 
however,  to  realize  that  traceability  is  not  a 
substitute  for  quality  in  the  design  process. 


Program  Elements 
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DESIGNING 

For  Mission  Assurance 

■ Performance  vs.  Insurance 

■ Test  Philosophy 

■ Government/Industry  Responsibility 

■ Software/Hardware  Trade 

■ Modeling 

■ Traceability /Quality 
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DESIGNING  FOR  MISSION  ASSURANCE 


Lee  L.  Farnham 

General  Manjger,  Space  Division 
General  Electric  Co. 


It  is  a pleasure  to  participate  as  a member  of  this 
panel  and  to  share  some  of  the  experience  ny  organ- 
ization has  acquired  while  living  in  the  changing 
world  of  mission  assurance  over  the  past  20  years. 

I have  focused  on  the  following  parts  of  the  con- 
ference's objective  — examination  of  key  elements 
and  factors  critical  to  space  mission  success,  and- 
identification  of  tools,  techniques,  and  methods  to 
minimize  and  control  risk. 

Vu-Graph  Programs 

General  Electric  has  participated  in  many  long-life 
mission  programs  for  NASA,  international  customers, 
and  DOD.  Some  of  them  are  shown  on  this  vu-graph. 
From  our  experience  on  these  programs,  we  have 
developed  a spacecraft  design  philosophy  for  mission 
assurance.  It  is  characterized  by  strong  adherence 
to  the  concept  of  Design  Verification,  and  the 
tailoring  of  test  plans  to  meet  the  individual  pro- 
gram's long-life  mission  requirements.  It  will  have 
time  only  to  outline  some  of  the  methods  we  have 
used  for  Design  Verification,  and  to  show  you  some 
of  the  tools  we  have  developed  to  help  us  "tailor" 
program  plans  to  Mission  performance  goals. 


The  majority  of  cur  early  programs  were  applications 
satellite  systems,  such  as  the  Nimbus  program,  begun 
in  the  early  1 960 ' s to  provide  data  for  atmospheric 
andand  meteorological  researchers.  As  a test  vehi- 
cle for  earth  observation  instruments,  the  program 
has  grown  in  capability  and  performance. 

Spacecraft  power  for  the  six  vehicles  already 
orbited  is  575  watts,  and  the  data  rate  is  800  kilo- 
bits. Total  spacecraft  weight  has  grown  from 
820  pounds  to  2,000  pounds  to  accommodate  growing 
instrumentation  payloads,  without  substantial  space- 
craft structural  redesign. 


The  program  required  3-axis  stabilization  for  pre- 
cise instrument  pointing,  reliable  command  control 
and  data  communication  systems,  and  an  orbital  life 
of  from  6 months  to  1 year.  In  service,  life  has 
ranged  from  32  months  for  Number  2 to  more  than 
99  months  for  Number  4.  The  last  three,  launched 
in  1970,  1972,  and  1975,  continue  to  return  useful 
data. 

Mission  assurance  was  a rather  primitive  discipline 
in  the  early  Nimbus  days.  Here  is  what  it  was  like 
in  that  era: 

• We  had  very  little  experience  or  data  to 
guide  our  design  effort. 

• Payloads  and  spacecraft  structures  were 
simple,  and  rather  small,  due  to  the 
limitations  of  the  booster  systems; 
therefore 

• We  didn't  have  payload  weight  for  exten- 
sive redundancy;  and 

• We  thus  had  to  endure  high  single-point 
failure  risks. 

These  limitations  in  knowledge  and  experience 
created  a heavy  reliance  on  analysis  techniques. 

Our  customer  also  had  limited  experience,  and  hoped 
to  reach  his  mission  assurance  goals  by  imposing 
rigid  specifications  on  his  contractors.  But  as 
the  early  programs  matured  and  design  teams  gained 
experience,  we  began  to  develop  a more  flexible  and 
tailored  design  process  aimed  at  meeting  mission 
requirements. 

I'd  like  to  trace  the  development  of  some  of  these 
techniques  on  successive  programs. 

In  the  beginning,  as  I said,  we  could  meet  the  rigid 
specifications  only  by  analysis.  In  order  to  verify 
the  analysis,  we  conducted  tests  at  every  possible 
level  of  assembly.  Naturally,  this  process  accu- 
mulated much  test  data.  But  it  did  nut  necessarily 
assure  mission  success,  since  we  were  testing  to 
meet  a specification,  without  the  benefit  of  actual 
long-life  mission  data.  But  it  was  this  test  data 
which  became  the  base  which  ultimately  allowed  us 
to  tailor  our  design  approach  to  particular  program 
requirements. 

The  Landsat  Program  which  followed  the  Nimbus  family 
of  spacecraft  was  designed  by  using  the  experience 
of  Nimbus.  Let  me  briefly  summarize  the  Landsat 
mission. 

The  Landsat  program  obtains  earth  resources  infor- 
mation on  a global  repetitive  basis  from  remote 
sensors  in  space. 

The  first  Landsat,  launched  in  1972,  was  designed  to 
operate  for  one  year  in  a near-polar,  sun- 
synchronous  orbit  570  miles  high.  Actual  life  was 
5 and  1/2  years,  after  which  it  was  turned  off  by 
NASA.  Landsat  2,  with  a design  life  of  one  year, 
was  launched  in  1975  and  after  three  years  is  still 
operating,  as  is  Landsat  3,  launched  last  month. 

The  Landsat  program  has  already  provided  a half- 
million  Images  of  the  earth  to  over  100  nations, 
using  wide-band  digital  data  transmission  systems 
to  accommodate  the  high  data  rates  necessary  for 


image  transmission.  These  images  provide  an  impor- 
tant input  in  monitoring,  mapping,  surveying,  and 
estimating  natural  and  man-made  resources,  such  as 
crops,  water  supplies,  energy  sources,  and  geologi- 
cal features. 

To  meet  mission  assurance  requirements  for  the 
Landsat  Program,  our  base  was  the  proven  designs 
from  the  Nimbus  family  of  spacecraft,  and  the  test 
procedures  we  used  to  verify  them.  The  designs 
include  the  3-axis  attitude  control  system,  the  open 
butterfly  spacecraft  configuration,  and  the  Solar 
Array  Paddle  system. 

To  meet  the  mission  requirements  for  new  component 
designs,  we  developed  and  applied  the  concept  of 
early  all-up  design  verification.  This  concept  was 
evolved  during  the  early  Nimbus  program  and  progres- 
sively substantiated,  for  we  found  by  experience 
that  System-level  Engineering  Model  tests  tended  to 
come  too  late  in  the  design  cycle  to  be  fully  effec- 
tive. The  engineering  model  also  proved  to  be  a 
poor  test  vehicle  in  the  formative  design  stage, 
since  the  test-point  access  was  very  difficult. 

Also,  replacement  and  repair  work  took  too  much 
time  and  effort  because  of  the  difficulty  in  dis- 
assembling and  reassembling  both  the  functional  and 
structural  components  for  troubleshooting  and 
adjustment. 


To  appreciate  these  difficulties,  look  at  this 
Engineering  Model  of  an  early  Nimbus  Spacecraft 
being  checked  during  systems  test.  Since  assembly 
normally  happens  very  late  in  the  design  cycle, 
there  is  no  provision  for  rapid  trouble  shooting 
and  easy  rework,  which  are  so  necessary  during  early 
tests  for  design  verification  of  the  overall  system. 


design  performance  and  create  the  opportunity  to 
make  an  intelligent  assessment  of  the  degree  of 
subsequent  system  testing  required. 


Bench  Integration  Test  is  designed  for  ease  of 
trouble-shooting  and  rework.  The  test  procedures, 
the  interfaces  with  the  ground  support  equipment, 
and  software  associated  with  the  system  are  impor- 
tant products  of  the  BIT  system,  since  we  maintain 
full  test  procedure  discipline  on  all  these  ele- 
ments. This  discipline  carries  through  the  assur- 
ance of  test  readiness  for  the  all-important  flight 
spacecraft. 

The  experience  we  gained  in  the  Earth  Observatory 
Spacecraft  has  proved  to  be  very  valuable  when 
applied  to  a communications  satellite  program 
called  BSE. 

BSE,  the  Medium-scale  Broadcast  Satellite  for 
Experimental  Purpose,  was  designed  by  General  Elec- 
tric under  a contract  to  Toshiba  for  the  National 
Space  Development  Agency  (NASDA)  of  Japan.  The 
spacecraft  was  successfully  launched  on  April  7 
into  a geosynchronous  orbit  where  it  will  be  used 
for  experimental  color  television  and  voice  trans- 
mission over  two  channels  to  small  ground  receivers 
in  remote  and  urban  areas  of  Japan.  These  experi- 
ments will  be  carried  out  over  a 3-year  period  to 
test  the  effect  of  atmospheric  and  geographic  char- 
acteristics on  the  Ku-Band  uplink  and  downlink 
satellite  signal  transmission. 

BSE  is  a long-life,  3-axis  stabilized  satellite  that 
weighs  approximately  1500  pounds  and  can  produce 
nearly  1 kilowatt  of  power  from  a sun-oriented  solar 
array. 


Contrast  this  with  the  technique  we  now  use  for 
early  system  design  verification.  We  call  it  the 
Bench  Integration  Test  or  "BIT"  system.  This 
approach  allows  an  early  all-up  system  test.  The 
components  are  laid  out  on  a specially  configured 
bench  that  allows  easy  access  to  components  through- 
out the  system.  We  have  tailored  the  system  test 
program  specifications  to  allow  the  use  of  the  BIT 
approach.  We  gain  early  verification  of  system 


On  this  program,  no  rigid  test  requirement  specifi- 
cation was  imposed  by  the  customer.  Rather,  he 
required  a specified  signal  level  and  quality,  and 
a mission  life  of  3 years.  This  allowed  us  to 
tailor  a Mission  Assurance  plan  to  meet  those 
requirements,  and  considerable  latitude  of  approach 
was  possible.  Proven  subsystems  were  used  exten- 
sively, and  new  payload  designs  were  verified  early 
through  the  BIT  process. 

Certain  critical  components  were  subcontracted.  I 
and  my  staff,  as  well  as  the  cognizant  design  team, 
participated  in  the  subcontractor  selection  process. 
Critical  components,  such  as  the  high-power  travel- 
ling wave  tube  amplifiers,  were  selected  with  spe- 
cial attention  to  long-life  performance,  as  well  as 
weight  and  power  dissipation.  We  determined  that 
management  involvement  at  all  levels  was  necessary 
to  assure  that  critical  components  from  subcontrac- 
tors would  meet  the  severe  long-life  requirement. 
This  level  of  management  Involvement  obtained  the 
needed  business  commitment  from  each  of  the  major 
subcontractors  to  meet  the  three-year  design 
requirement. 

Another  management  technique  which  grew  out  of 
Nimbus  and  Landsat  experience  was  very  successfully 
applied  on  BSE. 

We  call  it  the  EMQ  Design  team  approach.  The  "E“ 
designates  the  engineer  responsible  for  the  design 
of  the  component;  tne  "M"  identifies  the  manufac- 
turing specialist  whose  responsibility  is  produci- 
billty;  "Q"  is  for  the  Quality  engineer,  responsible 
for  the  controls  on  parts,  materials,  and  processes. 
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as  well  as  for  the  testability  of  the  component 
under  design.  These  teams  are  formed  at  the  begin- 
ning of  the  design  phase  and  provide  early  integra- 
tion of  the  disciplines,  so  necessary  for  the 
ultimate  performance  of  a newly  designed  product. 

It  is  this  team  that  is  "in  the  trenches"  to  prevent 
potential  single-point  failures  and  properly  analyze 
failure  reports.  It  is  this  team  that  answers  the 
following  questions  during  the  early  design  period: 

• Does  the  design  meet  the  requirements? 

• Is  it  producible? 

• Can  it  be  verified, by  test? 

We  feel  that  design  verification  is  so  important 
that  we  include  it  as  an  agenda  item  on  all  design 
reviews.  As  I mentioned  earlier,  the  all-up  BIT 
process  also  provides  an  early  verification  method 
for  the  ground  support  equipment,  software,  and  test 
procedures  associated  with  the  flight  spacecraft. 

Working  together,  the  EMQ  team  and  the  BIT  method 
provide  a design  that  can  be  verified  at  the  ear- 
liest possible  time  in  the  design  cycle  and  at  the 
most  logical  level  of  the  spacecraft  assembly. 

An  adjunct  to  this  technique  is  the  identification 
and  emplacement  of  the  skills  needed  to  manufacture 
and  test  the  components  requiring  the  use  of  new 
parts,  materials,  and  processes.  This  technique  is 
called  Process  Readiness,  and  it  functions  under  a 
rather  rigorous  plan  which  includes  management 
discipline. 

The  slide  shows  the  steps  by  which  a process  readi- 
ness plan  is  generated.  It  begins  with  a review, 
under  the  scrutiny  of  my  management  team,  to  deter- 
mine if  the  new  process  is  needed.  It  concludes 
with  a letter  to  the  General  Manager  which  certifies 
that  the  process  meets  the  criteria  established  by 
the  management  review  team.  You  can  see  the  signed 
specimen  letters  on  the  slide  by  which  my  managers 
literally  put  their  professional  reputations  on  the 
line,  in  writing,  to  guarantee  that  all  risk  and 
problems  have  been  removed  from  any  required 
processes. 

No  new  critical  process  is  approved  for  use  unless 
the  managers  of  Engineering,  Manufacturing,  and 
Quality  certify  in  writing  as  you  see  on  the  vu- 
graph.  I consider  this  management  involvement  the 
cornerstone  in  the  certification  of  Readiness  for 
every  new  Process. 

We  are  now  applying  these  techniques  to  a program 
which  has  the  most  demanding  requirement  for  mission 
assurance.  It  is  called  DSCS  III. 

The  DSCS  III  Communication  Satellite  program  will 
help  meet  national  military  communications  needs  of 
the  1 980 ' s . It  will  operate  at  synchronous  alti- 

tudes with  total  RF  output  power  of  120  watts 
through  6 channels.  The  RF  can  be  switched  through 
multiple  transmit  and  receive  antennas  to  provide 
coverage  patterns  ranging  from  the  whole  earth  to 
narrow  spot  beams.  Operational  versatility  is 
achieved  through  two,  19-beam  waveguide  lens  trans- 
mit antennas  and  a 61 -beam  waveguide  lens  receive 
antenna. 

The  telemetry  command  and  control  subsystem  inter- 
faces with  multiple  ground-based  terminals  and  is 


capable  of  providing  rapid  configuration  of  both 
transmit  and  receive  multi -beam  antennas.  The 
spacecraft  is  3-axis  stabilized  with  a 10-year 
design  life.  Latest  developments  in  hardening  are 
being  used  to  insure  reliability. 

In  operational  use,  a constellation  of  4 satellites 
will  be  placed  in  orbit  over  the  Atlantic,  East  and 
West  Pacific,  and  the  Indian  Ocean. 

I'll  have  time  for  only  a brief  summary  of  the  mis- 
sion assurance  tools  and  techniques  being  used  on 
this  program.  With  the  combined  requirement  of 
10-year  mission  life  and  radiation  hardening,  the 
greatest  impact  on  mission  assurance  was  determined 
to  be  at  the  piece-part  level.  This  determination 
has  dramatically  affected  the  procurement  specifica- 
tion  of  piece  parts. 


for  active  parts  evolved  as  the  mission  life 
requirement  increased  from  1 year  to  10  years.  For 
example,  in  the  1960  time  period,  the  military  JAN 
types  were  acceptable  for  the  mission  life  up  to 
1 year.  Next,  the  JAN  TX  line  demanded  parts  that 
were  screened  and  burned  in.  It  was  followed  by 
JAN  TXV  which,  in  addition,  required  "pre-cap" 
visual  Inspection  on  semiconductors  as  a tollgate  to 
minimize  workmanship  and  contamination  defects. 

These  provided  parts  specification  meeting  mission 
life  requirements  of  up  to  3 years. 

The  next  line  of  parts  specification  is  referred  to 
as  Class  A type,  which  requires  parts  to  be  examined 
by  the  Scanning  Electron  Microscope  as  well  as 
other  tests.  This  parts  specification  is  for  mis- 
sion life  up  to  5 years. 

At  present,  Industry  and  Government  are  in  the  early 
stages  of  developing  a Class  "S"  parts  line  for 
integrated  circuits  which  require  additional  con- 
trols and  certification  of  processes.  However,  the 
availability  of  Class  "S"  parts  specifications  is 
limited  and,  therefore,  we  at  General  Electric  have 
generated  our  own  Class  "S"  specification,  called 
an  "R"  drawing,  for  various  integrated  circuits, 
transistors,  and  diodes. 

Our  experience  has  also  shown  us  that  stress  testing 
at  all  levels  is  essential,  with  the  highest  stress 
applied  at  the  piece-part  level.  We  believe  that 
this  method  will  provide  the  high  reliability  parts 
required  to  meet  the  DSCS  III  10-year  mission. 
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In  this  way,  we  constantly  impose  new  and  more 
stringent  requirements  on  parts  and  materials  to 
meet  hardening  and  reliability  requirements.  But 
that  is  not  enough.  Additionally,  we  need  redundant 
subsystems,  and  these  require  additional  weight  and 
power,  imposing  new  economies  on  the  designer.  For 
these  economies,  we  use  hybrid  technology  exten- 
sively. The  net  result  is  a spacecraft  having  a 
tremendous  number  of  piece  parts  and  redundant 
critical  components.  We  can  tolerate  this  design 
principle  only  because  we  have  high  confidence  that 
we  have  isolated  the  risk  at  the  piece-part  level, 
and  that  our  stringent  p'ece-parts  test  program 
will  give  us  the  necessary  mission  assurance. 

It  is  very  clear  to  us  that  the  approaches  outlined 
and  followed  on  our  previous  programs  must  be 
expanded  for  future  missions. 

We  will  continue  to  use  the  BIT  philosophy  to  con- 
trol the  design  process.  By  achieving  design  veri- 
fication at  the  earliest  possible  time  in  the  cycle, 
we  also  prevent  the  unnecessary  proliferation  of 
testing  farther  downstream.  We  believe  that  our 
experience  in  achieving  successful  flight  perfor- 
mance has  shown  that  testing  can  be  more  effective 
when  tailored  to  the  individual  needs  of  a program 
rather  than  working  from  a standard  baseline  speci- 
fication. For  future  missions,  these  methods  may 
allow  more  creative  and  cost-effective  solutions  as 
a positive  way  of  addressing  the  unique  requirements 
of  each  program. 

I believe  that  succeeding  generations  of  spacecraft 
will  be  called  upon  to  meet  higher  performance 
standards  on  missions  which  will  increase  in  com- 
plexity. Concurrently,  we  will  see  demands  for 
even  longer  orbital  life. 

This  will  impose  two  requirements  on  the  designer. 
First,  he  must  establish  a family  of  piece-part 
specifications  and  selection  processes  which  enable 
him  to  match  his  assurance  level  to  the  demands  of 
each  mission;  and  second,  on  this  foundation  he 
must  erect  a system  structure  which  will  verify  his 
designs  from  top  down  early  in  the  design  process. 

At  General  Electric  our  Mission  Assurance  plan  is 
built  on  these  assumptions  and  operates  through  five 
basic  principles: 

First,  we  believe  in  earliest  possible  all-up  system 
integration  testing  because  we  have  learned  that 
this  is  the  effective  and  most  economical  level  at 
which  to  discover  and  remedy  system-oriented 
problems. 

Second,  we  exert  constant  pressure  on  piece-part 
qualification  and  selection,  driving  our  test 
regimes  steadily  downward  to  the  most  elemental 
level  in  order  to  meet  the  challenges  of  constantly 
Increasing  mission  life. 

Third,  we  believe  that  the  EMQ  team  design  process 
on  each  component  should  incorporate  necessary 
standards  for  producibility  and  testability.  There 
is  no  place  in  the  modern  design  cycle  for  proce- 
dures which  attempt  to  Inject  these  elements  after 
the  fact. 

Fourth,  an  effective  Mission  Assurance  program 
demands  not  only  Management  oversight,  but  Manage- 
ment involvement.  This  will  assure  that  the  level 


of  Management  attention  is  commensurate  with  the 
mission  risk  involved  at  each  stage  of  the  design 
and  development  process. 

Fifth,  there  must  be  a constant  search  for 
innovation  and  flexibility  in  both  design  and  test 
principles.  The  designing  organization  must  be 
given  every  opportunity  to  apply  its  experience  and 
ingenuity  to  produce  the  largest  Mission  Assurance 
increment  at  the  lowest  possible  cost. 
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Abstract 

Adequate  testing  is  one  of  the  key  elements  in 
meeting  Mission  Assurance  Requirements.  The  Govern- 
ment and  the  contractor  must  give  more  emphasis  to 
the  role  of  testing.  All  test  elements  must  recog- 
nize the  importance  of  Mission  Assurance.  Normal 
development  testing  and  operational  testing  is  not 
sufficient.  A total  System  Test  Bed  is  necessary 
to  assure  performance. 

Introduction 

I have  chosen  to  focus  my  remarks  on  the  Role  of 
Testing  in  Designing  for  Mission  Assurance.  How- 
ever, before  I get  to  the  main  theme,  there  are 
several  comments  I would  like  to  make.  I want  to 
touch  on  the  contractors'  role,  and  the  importance 
of  the  Government's  role. 

As  a contractor,  we  face  the  problem  of  balancing 
Performance,  Cost  and  Schedule.  But  we  are  not 
autonomous  the  Government  must  structure  a pro- 
gram compatible  with  an  appropriate  balance,  must 
communicate  their  requirements,  and  must  be  party 
to  the  decisions  necessary  to  maintain  the  balance. 

I cannot  emphasize  too  heavily  the  importance  of  a 
clear  understanding  of  the  total  program  require- 
ments. While  the  contract  specification  must  be 
the  baseline,  these  documents  can  only  scratch  the 
surface  in  conveying  the  entire  picture.  Other 
contract  communications , such  as  contract  incen- 
tives, often  reflect  on  the  implementation  of  the 
program  in  such  a fashion  that  Mission  Assurance  is 
weakened.  Too  often,  recently,  contract  incentives 
have  not  been  properly  used  as  a tool  for  establish- 
ing the  relative  Importance  of  requirements  (per- 
formance, schedule  and  cost).  It  results  in 
motivating  the  contractor  to  maximize  his  profit, 
often  to  the  detriment  of  the  program.  More  empha- 
sis must  be  placed  on  balancing  the  program  objec- 
tives by  rewarding  the  contractor  for  meeting  the 
mission  performance  requirements. 

Mission  Assurance  is  not  a separable  requirement: 

It  is  an  essential  part  of  the  performance  require- 
ments. Its  role  must  be  recognized  in  structuring 
the  program.  Adequate  allowance  must  be  made  in 
the  schedule  and  in  the  costs  to  assure  mission 
success.  Adequate  testing  is  one  of  the  key  design 
elements  that  has  a significant  influence  on  meet- 
ing the  Mission  Assurance  requirements,  and  on  total 
cost  and  schedule  Impact.  Thus,  we  (the  Government 
and  the  Contractor  jointly)  need  better  balancing 
of  performance,  cost  and  schedules  tied  into  the 
testing  function  for  Mission  Assurance  attainment. 


today's  world,  the  emphasis  in  electronics  is  more 
and  more  on  careful  design,  analysis  and  review,  to 
the  detriment  of  testing.  GTE  Sylvania's  success- 
ful experience  indicates  that  there  is  no  substitute 
for  adequate  testing  to  assure  Mission  Success. 

Now  let  us  examine  this  broad  subject  - Testing. 

The  Winter  1977  Defense  Systems  Management  Review 
was  devoted  to  test  and  evaluation.  Many  of  the 
contributors  discussed  testing  to  assure  hardware 
reliability.  The  role  of  testing  to  provide  Mission 
Assurance  testing  on  a systems  level  was  hardly 
mentioned.  One  contributor  to  the  Defense  Systems 
Management  Review  defines  the  primary  responsibili- 
ties of  development  test  as  answering  the  two 
questions: 

1.  "Does  the  system  meet  the  engineering 
specifications?  and, 

2.  Is  the  system  sufficiently  developed  to 
enter  production?" 

The  same  author  says: 

"Operational  test  and  evaluation  issues  are: 

a.  Does  the  system  operate  effectively  in 
its  intended  operational  environment? 
and, 

b.  Can  "blue-suit"  personnel  maintain  the 
system  as  .-envisioned  in  the  maintenance 
concept?"  ll  J 

Neither  the  development  test  objectives  nor  the 
operational  test  objectives  include  the  total  sys- 
tem tests  to  assure  mission  success. 

Another  contributor  lists  his  goals  as: 

"•  Shorter  tests. 

• Schedules  where  efforts  can  be  combined 
and  outlays  reduced. 

• Ways  testers  can  work  with  industry  and 
developers  to  further  reduce 
expenditures. "t^] 

While  these  are  fine  goals,  how  about  better 
assurance  of  mission  success?  I advocate  It  is 
more  technically  sound,  cheaper,  and  faster  to 
solve  the  Mission  Assurance  design  problems  during 
♦he  initial  design/development  program,  utilizing 
a total  test  program  than  to  discover  these  design 
anomalies  during  the  complex  field  test  programs  or 
under  operational  use. 


Identifying  the  Tests 

What  are  sow  of  these  design  tests  that  are  key  to 
supplying  Mission  Assurance? 


The  Importance  of  Testing  for  Mission  Success  Engineering  Breadboard  Tests 


A strategic  weapon  system  must  be  capable  of  meeting 
all  of  its  operational  requirements  under  actual 
field  conditions  and,  in  many  cases,  this  may  mean 
sitting  in  the  readiness  mode  forever.  Testing  is 
one  of  the  design  tools  necessary  to  assure  that  it 
will  do  the  job,  if  needed.  Unfortunately,  in 


These  are  the  earliest  design  tests  necessary  to 
support  the  design  of  the  equipment  and  ferret  out 
inherent  design  problems.  They  should  not  be  overly 
structured,  but  it  is  important  that  they  be  suffi- 
ciently documented  to  provide  a basis  for  review 
and  understanding  by  others. 
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Engineering  Development  Tests 

Functional  elements  of  the  system  should  be  tested 
before  the  system  design  is  truly  frozen  - it  is  in 
these  tests  that  operational  anomalies  not  covered 
by  the  specifications  are  first  identified:  i.e., 
is  there  an  unexpected  harmonic  content?  This  data 
must  then  be  fed  into  the  system  design  to  assure 
that  no  degradation  of  performance  will  result. 

Prototype  Tests 

Prototype  tests  confirm  the  design  and  assure  that 
the  final  manufactured  units  will  perform  in  accor- 
dance with  the  design  objectives.  Prototype  tests 
should  be  conducted  at  all  levels: 

a.  Components  - Components  must  be  tested  at 
two  levels;  they  must  be  tested  to  verify 
their  performance  under  the  expected 
application,  and  they  must  be  tested  to 
assure  their  inherent  quality.  The 
second  requirement  is  the  normal  quality 
control  function.  Component  testing  is 
returning  to  its  proper  balance. 

b.  Assemblies  and  Subassemblies  - You  will 
find  many  programs  that  get  into  diffi- 
culties have  short-changed  this  design 
step.  Prototype  assemblies  and  sub- 
assemblies  should  be  returned  to  the 
engineers  to  verify  that  these  units  work 
like  the  breadboard.  Here  is  where 
problems  with  ground  loops,  stray  coupl- 
ing, and  thermal  problems  really  surface. 
Many  of  these  will  have  been  avoided  if 
the  breadboard  tests  are  thorough,  but 
this  is  the  place  to  catch  the  rest  of 
them  before  the  production  run. 

junctional  Entity  Tests 


Most  systems  are  subsystems  made  up  of  major  func- 
tional entities.  In  our  case,  these  are  usually 
Contractor  End-Items  and  constitute  a single  rack 
of  equipment.  These  have  a complete  Government 
procurement  specification  and  should  be  tested 
against  their  specification.  In  addition  to  the 
functional  requirements,  this  is  the  level  at  which 
environmental  and  quality  tests  can  be  performed 
most  economically.  Unfortunately,  the  pressures  of 
schedule  and  costs  often  delay  the  performance  of 
the  complete  qualification  tests.  For  this  reason, 
it  is  Imperative  that  as  much  data  as  possible  be 
obtained  on  environmental  problems  earlier  in  the 
design  program,  utilizing  an  engineering  test  pro- 
gram, especially  if  the  formal  qualification  test 
program  has  been  eliminated  or  delayed  into  the 
production  phase. 

Subsystem  Test 

The  highest  level  for  which  the  individual  contrac- 
tors are  usually  responsible  is  the  subsystem  test- 
ing. Too  often  it  is  assumed  that  stringest  testing 
at  the  Customer  End-Item  level  is  an  adequate  sub- 
stitute for  a thorough  subsystem  test.  When  the 
pressure  from  cost  and  schedule  builds.  It  results 
in  pressures  to  reduce  or  eliminate  the  subsystem 
testing.  If  a high  confidence  In  the  Mission 
Success  is  to  be  achieved,  the  subsystem  must  be 
tested  far  beyond  those  tests  necessary  to  confirm 
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compliance  with  the  stated  normal  system  operational 
requirements.  There  must  be  tests  to  confirm  all 
operational  uses  of  every  part  of  the  system, 
especially  those  uses  or  modes  which  occur  only 
occasionally  in  pre-attack  normal  operations,  but 
are  critical  during  post-attack  operations.  The 
identification  of  the  parameters  to  be  measured  is 
in  fact  a significant  design  task.  The  failure 
mode  and  effect  analysis  provides  a guide  in  deter- 
mining key  parameters,  but  properly  designed  testing 
is  a specialty  unto  itself. 

The  tests  must  be  designed  to  detect  every  anomaly 
and  must  capture  the  data  to  pinpoint  the  cause. 
Mission  Assurance  can  be  achieved  only  if  the  ano- 
malies are  identified,  and  the  root  cause  deter- 
mined. Only  after  the  cause  is  identified  can 
appropriate  corrective  action  be  taken.  I don't 
mean  to  imply  that  every  unexpected  event  is  wrong 
often  it  may  be  normal  and  no  further  action  will 
be  necessary;  however,  this  decision  can  be  made 
only  after  the  root  cause  is  understood.  The  sub- 
system test  program  must  provide  also  the  flexibil- 
ity for  special  tests  where  unexpected  events 
dictate  a need.  It  is  alarming  how  many  major  pro- 
grams have  inadequate  subsystem  test  programs! 

System  Test 

Everything  that  can  be  said  about  subsystem  testing 
can  equally  be  said  for  total  system  testing  - and 
more  so!  In  today's  digital  world,  the  complexity 
of  the  interfaces  has  mushroomed.  . . we  see  one 
contractor's  computer  controlling  another  contrac- 
tor's hardware,  perhaps  with  a third  contractor's 
software  residing  in  the  computer.  Simulators  are 
not  the  answer!  A complete  system  using  the  real 
equipment  is  the  best  guarantee  of  performance! 

For  most  major  systems,  it  is  more  cost  and 
schedule-effective  to  perform  the  initial  system 
tests  program  at  the  contractor's  design  facility, 
rather  than  totally  performing  system  tests  at  the 
Field  Sites. 

Significant  Elements  of  a Proper  System  Test 

The  significant  elements  of  a proper  system  test 
are: 

1.  All  equipment  must  be  as  close  to  the 

final  hardware  configuration  as  possible. 

A good  test  bed  will  contain  the  com- 
plete suite  of  operational  hardware.  All 

means  ALL  and  it  includes: 

• Operational  Cabling 

• Operational  Environmental  Control 

• Operational  Power  Systems 

• Representative  Radio  Links 

• Representative  Cable  Links 

• A Source  of  Representative  Traffic 


2.  The  Test  Bed  must  be  as  close  to  the  final 
operational  equipment  layout  as  is  possi- 
ble. Many  more  problems  are  associated 
with  physical  layout  than  are  normally 
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recognized  by  those  who  are  functionally 
oriented.  Only  with  a good  mechanical 
simulation  will  you  be  able  to  identify: 

• Clearance  problems  - It  can  cost  more  to 
change  from  a straight-thru  connector 
to  a right-angle  connector  than  it 
costs  to  modify  a card. 

• Thermal  problems 

• Cable  lengths 

• Human  factors 

• Maintenance  and  access 

• Operator  abuse  - (Yes,  we  have  had  to 
correct  a critical  problem  because  the 
operator's  chair  slammed  into  the 
console. ) 

3.  Strict  Configuration  Control  is  a must. 
This  statement  has  to  be  applied  to  both 
software  and  hardware. 

4.  Test  documentation  must  be  complete. 

5.  The  Systems  Engineers  must  prepare  Test 
Requirements  which  must  encompass  100  per- 
cent of  the  Mission  Operational 
requirements. 

6.  The  test  operation  should  be  independent 
of  the  design  groups.  Their  job  is  to 
gather  the  data  and  define  the  results, 
not  to  continue  the  design  or  to  ration- 
alize away  the  results  as  not  important. 

7.  Adequate  design  and  system  engineering 
support  from  all  Associate  Contractors 
must  be  available.  While  it  is  the  role 
of  the  test  operators  to  run  the  tests, 
the  designers  are  in  the  best  position 
to  recognize  the  significance  of  any 
indications  and  correct  deficiencies  in 
the  tests  on  the  spot. 

8.  Fast  accurate,  feedback  of  results  to 
ALL.  Along  with  this  goes  the  responsi- 
bility of  the  contractors  to  review  the 
results  and  take  action. 

9.  A representative  of  the  procuring  agency 
must  be  a part  of  the  test  team.  The 
operation  of  test  programs  requires  many 
decisions  which  are  outside  the  responsi- 
bility of  the  test  operator,  priorities 
must  be  changed,  an  equipment  malfunction 
may  preclude  running  a test,  or  mean  a 
modification  to  the  test  plan.  Schedule 
slips  may  impact  on  plans,  and  support 

to  a high  priority  problem  may  throw 
everything  Into  a cocked  hat. 

This  type  of  initial  System  Test  Program  often  gets 
the  first  axe  by  many  Covernment  SPOs  and/or  con- 
tractors because  of  the  design  costs  and  schedules, 
even  if  it  is  realized  that  It  will  take  greater 
time  and  dollars  later  to  resolve  the  anomalies. 

Does  This  Test  Design  Tool  Work? 

Our  experience  Indicates  it  Is  a huge  success  If 
planned,  budgeted  and  implemented  under  the  same 


management  and  customer  leadership  that  all  tech- 
nical and  contractual  requirements  are  directed. 

We  applied  this  approach  to  the  Minuteman  Weapon 
System.  For  the  WS-133B  Minuteman  System  Develop- 
ment Contract,  the  Government  had  balanced  incen- 
tives for  performance,  test,  schedule  and  cost 
items.  Thus  GTE  Sylvania,  as  the  design,  develop- 
ment, build  and  test  contractor,  was  able  to  estab- 
lish its  own  proven,  cost-effective  design/test 
balanced  program,  including  a System  Test  Bed. 

A configured  launch  facility  and  a configured  launch 
control  facility  were  established  in  nearby  Waltham, 
Massachusetts  and  a second  launch  control  facility 
was  established  locally  in  West  Roxbury  to  provide 
a radio  link  as  part  of  the  total  test  configura- 
tion. System  testing  started  in  1963.  After  its 
initial  basic  use,  this  System  Design  Test  Bed  was 
successfully  utilized  to  support  the  1965-1967  VAFB 
firings  and  the  Operational  Deployment  during  1965- 
1967.  Additionally,  it  has  been  used  from  1969 
until  now  for  resolving  design  modifications  for 
three  major  weapon  system  additives  (UHF,  Hardness 
and  Command  Data  Buffer).  The  responsibility  for 
System  Testing  is  now  being  transitioned  to  OALC  at 
Hill  Air  Force  Base.  The  equipment  from  the  GTE 
Sylvania  Test  Beds  is  being  used  to  establish  the 
HETF-II  Test  Bed  there. 

It  is  difficult  to  prioritize  the  many  uses  these 
test  beds  have  served.  The  emphasis  has  changed  as 
the  system  has  matured.  The  mission  of  the  test 
bed  was  defined  as: 

• Demonstrate  that  integrated  equipment  and 
software  meet  the  Weapon  System 
requirement. 

• Identify  design  problems  early,  thereby 
permitting  least-costly  correction. 

• Duplicate  field  problems  and  verify 
recommended  solutions. 

• Provide  a test  vehicle  for  kit-proofing. 

• Test  improvements  to  the  system. 

Did  the  test  program  fulfill  its  mission? 

Let's  look  at  a few  statistics  (for  the  initial 
WS-133B  System): 

794  problems  were  identified  by  testing  the 

integrated  system. 

265  of  these  results  in  GTE  Sylvania  ECPs. 


Because  of  the  early  Identification  of  these  prob- 
lems, 65  percent  of  the  ECP  changes  were  incorpor- 
ated in  line  production  prior  to  fielding  of  the 
operational  sites. 

If  the  ECPs  had  required  field  retrofit,  the  cost 
would  have  been  an  additional  $20  million.  The 
cost  for  running  the  test  beds  during  this  period 
was  $5  million.  A net  saving  to  the  Government  of 
$15  million  - and  this  is  only  the  saving  on  the 
GTE  Sylvania  equipment:  similar  benefits  were 
derived  by  the  other  associates.  Of  even  greater 
significance  no  missile  launches  were  delayed  or 
failed  because  of  a ground  launch  failure.  Total 
Mission  Success! 
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I want  to  emphasize  with  an  example  that  the  func- 
tion of  a test  bed  goes  far  beyond  hard  dollar 
savings  and  the  hard  failures  of  hardware.  During 
our  system  testing  at  Waltham,  a critical  loss  of 
system  synchronization  was  identified.  This  was  a 
subtle  problem  that  occurs  only  when  the  system  is 
in  a partially  "down"  state.  That  is  a key  - in 
normal  operation,  the  problem  does  not  exist. 
Operational  testing  would  not  have  identified  it. 
Testing  at  Vandenberg  failed  to  identify  it.  A 
special  team  was  established  with  representatives 
from  GTE  Sylvania,  Autonetics,  Univac  and  TRW.  It 
was  determined  that  a hardware  interrupt  in  the 
Weapon  System  Computer  was  at  fault  and  a Univac 
ECP  resulted.  This  type  of  system  capabilityexists 
only  at  a system  test  bed  and  must  be  an  integrated 
part  of  assuring  Mission  Success. 

In  summary,  I would  like  to  emphasize  the  following 
points: 

• The  Government  and  the  Contractors  must 
give  more  recognition  to  the  role  of 
testing. 

• Adequate  testing  means  testing  at  all 
levels. 

• Testing  should  include  the  entire  system. 

• Testing  is  cost-effective. 

• Testing  is  necessary  to  assure  perfor- 
mance throughout  the  life  of  a program. 

e Adequate  testing  is  a key  element  in 
designing  for  Mission  Assurance. 
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DESIGN  FOR  MISSION  ASSURANCE 
Robert  W.  Hager 

DIVISION  MANAGER,  SMSD,  BOEING  CO. 

The  success  of  a space  mission  is  primarily  estab- 
lished during  the  design  phase  of  the  program. 
Although  subsequent  activities  in  procurement, 
manufacturing  and  testing  are  critical  and  can  im- 
pact mission  success  if  not  properly  conducted, 
there  is  no  way  to  build  back  in  reliability  if  it 
doesn't  exist  in  the  basic  design.  Therefore, 
achieving  mission  success  must  be  a primary  design 
goal . 

There  are  a number  of  design  techniques  which  can 
significantly  improve  the  mission  success.  These 
include;  (1)  more  emphasis  on  parallel  operating 
redundancy.  That  is,  redundancy  without  switching 
to  a standby  system;  (2)  use  of  a conservative 
electronic  part  de-rating  criteria;  (3)  have  the 
designer  directly  involved  in  the  failure  mode  and 
effect  analysis;  (4)  eliminate  single  points  from 
the  design  as  far  as  possible  and  rigidly  control 
those  which  cannot  be  eliminated;  (5)  utilize  sneak 
circuit  analysis  to  assure  that  there  are  no  un- 
known and  unwanted  feedbacks  in  the  system;  and 
(6)  specify  conservative  burn-in  testing  require- 
ments. 

All  of  these  techniques,  I'm  sure,  sound  familiar 
and,  to  some  extent,  are  being  used  in  all  complex 
designs.  In  the  design  of  the  Inertial  Upper  Stage 
we  are  using  these  techniques  with  some  unique 
applications.  I will  cover  some  of  these  in  more 
detail  and  show  the  benefit  we  expect  to  gain. 

One  of  the  more  significant  questions  which  arises 
in  the  design  is  how  much  redundancy  is  really  cost 
effective.  We  can  add  more  redundant  elements  in 
the  system  and  continue  to  improve  to  some  degree 
the  calculated  reliability,  but  with  an  increasing 
cost  and  weight  penalty.  Optimizing  the  system 
architecture  is  a process  of  balancing  the  cost  of 
reliability  improvements  against  the  cost  of  fail- 
ures to  achieve  minimum  total  cost  over  the  life- 
time of  the  program,  as  shown  in  Figure  1.  The 
approach  starts  with  a single  thread  system  and 
adds  reliability  options.  The  cost  of  failures  is 
evaluated  for  each  option  by  multiplying  the  ex- 
pfted  number  of  failures  by  the  cost  of  replace- 
ment or  repair  and  re-flight  and  then  is  summed 
with  the  life  cycle  cost  of  the  option  to  obtain 
total  program  cost.  This  process  of  adding  options 
continues  until  a minimum  program  cost  point  is 
located.  There  is  generally  a rather  sharp  upswing 
beyond  this  point  because  the  life  cycle  costs  of 
additional  redundancy  become  increasingly  larger 
and  the  savings  in  cost  of  failures  is  smaller  as 
the  mission  reliability  increases. 

The  IUS  Program  cost  analysis  model,  as  shown  in 
Figure  2,  is  the  method  used  to  guide  the  design 
toward  a cost  optimized  system  architecture.  Be- 
cause of  the  high  leverage  that  the  IUS  has  on  the 
total  cost  of  successfully  delivering  spacecraft  to 
orbit,  the  cost  effectiveness  is  conducted  at  a 
level  above  the  IUS  hardware  life  cycle  cost  and 
Includes  the  cost  of  replacing  aborted  or  lost 
missions.  The  model  is  run  for  each  reliability 
option.  For  each  separate  mission,  the  model  sums 
the  mission  cost  elements.  It  then  determines  the 
probability  of  an  abort  prior  to  IUS  deployment 


from  the  orbiter  as  well  as  the  cost  of  reflying  the 
mission  after  refurbishment  of  the  recovered  IUS/ 
spacecraft.  It  also  determines  the  probability  of 
an  IUS  mission  loss  after  leaving  the  orbiter.  The 
model  processes  the  total  112  projected  missions 
and  sums  the  total  cost  for  the  particular  configu- 
ration option. 

The  results  of  the  cost  optimization  are  shown  in 
Figure  3.  Starting  with  the  single  thread  system 
with  a reliability  of  0.84,  redundant  elements  were 
added  to  the  system  in  a prioritized  order  of  those 
with  the  largest  reliability  component  per  dollar 
of  life  cycle  cost.  The  plot  is  in  reality  a series 
of  point  options.  The  cost  optimum  reliability  was 
achieved  with  the  addition  of  redundant  drives  on 
each  of  the  four  thrust  vector  control  actuators. 
Additional  redundant  subsystems  such  as  the  third 
computer  were  not  cost  effective  because  the  addi- 
tional cost  was  higher  than  the  savings  in  cost  of 
probable  failure.  A number  of  additional  redundant 
items  were  also  evaluated  and  not  used.  The  limit 
is  the  single  point  reliability  of  the  solid  rocket 
motors  and  the  structure.  In  the  case  of  the  IUS, 
the  cost  optimized  point  exceeded  the  .96  relia- 
bility requirement.  There  may,  however,  be  other 
programs  where  this  type  of  analysis  would  show  an 
optimum  point  below  a pre-selected  reliability 
requirement.  In  that  case,  the  reliability  require- 
ment hould  be  re-evaluated.  The  results  of  this 
method  are  relatively  insensitive  to  the  cost  of 
the  spacecraft.  For  example,  a 50  increase  in 
spacecraft  cost  would  Increase  the  program  costs 
by  only  1.3  . Very  conservative  estimates  of  the 
reliability  for  individual  elements  derived  from 
MIL  Handbook  217B  are  used  in  these  calculations 
for  several  reasons.  One,  the  real  life  failure 
rates  are  probably  higher  than  the  levels  shown  in 
the  handbook.  The  number  of  units  will  be  too 
small  to  obtain  real  reliability  data  and  a more 
conservative  analysis  approach  is  desirable.  Also, 
a conservative  estimate  was  desired  in  the  design 
as  once  the  architecture  was  established,  further 
redundancy  is  eliminated.  The  results,  however, 
are  not  very  sensitive  to  the  individual  element 
reliability  as  the  significant  change  in  relia- 
bility results  from  redundancy. 

The  resulting  system  architecture  selected  is  shown 
in  Figure  4.  The  only  single  point  elements  are 
the  star  scanner  optics,  the  reaction  control  sys- 
tem supply,  the  solid  rocket  motors  (SRMs),  and  the 
structure.  The  redundant  IMU  consists  of  three 
groups  of  Instruments.  Two  of  the  three  groups  con- 
tain two  gyros,  two  accelerometers  and  associated 
electronics.  The  third  group  contains  one  gyro 
and  one  accelerometer  and  associated  electronics. 

The  internal  physical  arrangement  and  functional 
redundancy  is  such  that  any  three  of  five  gyros 
and  any  three  of  five  accelerometers  are  adequate. 

Avionics  consists  of  two  parallel  strings  confi- 
gured so  that  either  string  can  perform  all  com- 
puter and  control  functions.  Full  cross  strapping 
at  different  points  in  these  strings  was  not  found 
to  be  cost  effective.  A redundant  telemetry  and 
communication  system  was  not  cost  effective  because 
of  a redundant  self-actuation  circuit.  Redundant 
thrusters  are  provided  at  each  of  six  locations. 

Each  thruster  has  two  redundant  solenoid  valves. 
Redundant  separation  nuts  are  provided  at  eight 
locations  such  that  either  of  the  two  nuts  will 
allow  separation  of  the  stages.  Redundant  safe 
and  arm  devices  with  parallel  explosive  trains  to 
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the  igniter  are  provided  on  each  solid  rocket 
motor. 

Conservative  de-rating  of  each  electronic  part  is 
the  most  cost  effective  way  to  achieve  higher  re- 
liability in  the  design.  Both  electronic  and 
thermal  stress  de-rating  are  necessary.  Figure  5 
shows  four  typical  examples  of  de-rating  used  in 
the  I US  design.  Use  of  the  allowed  voltage  tem- 
perature de- rating  for  capacitors,  as  an  example, 
will  result  in  77%  reduction  of  the  part  failure 
rate.  Individual  part  de-rating  is  accomplished 
during  the  design  process.  The  electrical  stress 
is  calculated  and  parts  are  selected  to  be  within 
the  allowed  de-rating.  The  part  operating 
temperature  is  estimated  by  thermal  analysis  of 
the  package.  Where  necessary,  the  thermal  environ- 
ment is  reduced  to  the  allowable  range  by  providing 
heat  conduction  paths.  Where  de-rating  can't  be 
accomplished  in  a reasonable  manner,  a cost  trade 
is  required.  The  cost  of  an  alternate  design 
versus  the  impact  on  reliability  of  the  higher 
temperature  or  electrical  environment  can  be  eval- 
uated using  the  cost  model.  The  actual  part  thermal 
and  electrical  environment  are  then  verified  during 
development  and  qualification  tests. 

A good  failure  mode  and  effects  analysis  (FMEA) 
process  involves  direct  participation  by  the  de- 
signer. This  is  essential  as  no  one  understands 
the  design  circuitry  and  functions  better  than  the 
designer.  System  reliability  personnel  should  pro- 
vide additional  input  and  support,  including  overall 
FMEA  structure,  critiquing  the  designers  analysis 
to  make  sure  it  is  failure  oriented  rather  than 
success  oriented,  and  determining  system  level 
failure  modes  and  probabilities. 


Another  action  during  the  desian  phase  is  to  plan 
the  production  reliability  verification  testing. 

The  burn-in  test  hours  and  environment  conditions 
such  as  vibration  and  thermal  cycling  must  be  suf- 
ficient to  cause  the  infant  mortality  defects  to 
be  hard  failures.  An  adequate  burn-in  test  program 
can  be  an  effective  approach  to  reducing  the  system 
failure  ro’e. 

The  effectiveness  of  the  50  hour  burn-in  test  for 
SRAM  avionics  hardware  sets  is  shown  in  Figure  6. 
Fewer  than  one  in  100  units  failed  during  the  last 
hour  of  testing  versus  over  one  in  ten  units  during 
the  first  hour  of  the  test.  Both  thermal  cycling 
and  vibration  were  imposed  on  the  equipment  during 
the  burn-in  testing. 

Another  technique  which  can  significantly  improve 
the  mission  success  is  the  use  of  sneak  circuit 
analysis  of  hardware  and  software  and  modifying  the 
design  as  necessary.  Sneak  analyais  uncovers  latent 
or  sneak  circuit  conditions  that  would  inhibit  de- 
sirect  functions  or  cause  undesired  functions  to 
occur  without  a component  having  failed.  The  ana- 
lysis provides  an  independent  check  of  the  design 
in  that  it  identifies  single  failure  points,  un- 
necessary logic  or  components  or  improper  applica- 
tion of  redundant  paths.  The  method  involves  a 
computer-aided  development  of  the  design  circuitry 
into  simplifed  network  trees  which  can  be  analyzed 
to  identify  sneak  paths,  or  sneak  timing.  Several 
spacecrafts  have  had  sneak  circuit  analysis  con- 
ducted on  specific  subsystems.  On  the  global  posi- 
tioning system,  the  subsystems  evaluated  included 
the  electrical  power,  control  circuits,  interfaces 
with  the  digital  logic  and  logic  analysis  of  the 
dual  command  decoder . 


To  the  greatest  extent  practical,  single  point 
failures  identified  by  the  FMEAs  should  be  elimi- 
nated from  the  design  by  providing  redundant  paths. 
Single  points  are  allowed  in  the  IUS  design  only 
if:  (1)  redundancy  to  eliminate  the  single  point 
is  not  possible  or  practical  such  as  the  solid 
rocket  motors;  or  (2)  the  probability  of  a single 
point  failure  is  low  and  redundancy  to  eliminate 
it  is  not  cost  effective,  such  as  staging  connec- 
tors. 

The  justified  single  points  are  evoluated  against 
the  following  registration  criteria:  Does  the 
part  have  operating  or  shelf  life  limitations?  Is 
it  difficult  to  manufacture  or  procure?  Does  it 
have  a known  poor  history?  Is  there  a lack  of  re- 
liability history?  Has  a processing  deficiency 
either  been  experienced  or  anticipated? 

If  the  item  meets  one  or  more  of  the  criteria,  it 
is  entered  on  the  registered  item  list.  A regis- 
tered item  control  plan  is  prepared  and  implemented 
for  the  item. 

This  plan  would  include,  as  necessary,  additional 
in-process  inspections,  additional  mterials  and 
process  controls,  additional  testing  (both  destruc- 
tive and  non-destructive),  special  part  handling, 
tracability  of  each  part  and  where  all  others  of 
the  type  and  lot  are  located  and  use  of  further 
conservative  de-rating.  This  approach  should  con- 
trol the  parts  and  further  reduce  the  failure  pro- 
bability. 


The  analysis  identified  a sneak  path  that  would 
cause  the  spacecraft  to  tumble.  Redundant  thrusters 
would  fire  when  firinq  a primary  thruster  in  a 
different  axis.  On  the  application  technology 
satellite  a condition  was  identified  in  the  auto- 
matic deployment  sequence  that  could  cause  simul- 
taneous firing  of  two  or  more  squibe  which  could 
have  resulted  in  loss  of  the  spacecraft. 

In  conclusion,  the  design  phase  is  the  time  to  build 
in  the  mission  reliability  and  there  are  techniques 
available  which,  if  consistently  and  effectively  used, 
can  significantly  improve  mission  success. 


System  Architecture  Optimization 
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Richard  Schwartz 

Vice  President  and  Division  Director 
of  Satellite  Systems 
Rockwell  International 

Designing  For  Mission  Assurance 
at  Subcontractor 

• Management 

a Contract  Type 

• Schedule 

• PDR  Period 

• CDR  Period 

• Prototype  Hardware 

• Hardware  Fabrication 

• Test 

• Shuttle 

Management 

• Establish  single  point  for  contract 

management  at  prime 

• Engineering  vs.  contracts 

• Brief  total  subcontract  team  on  where  his 
piece  fits  in  the  total  program 

• Visits 

• Often  by  key  members  of  management 

• Technical  assistance 

• Some  areas  must  be  provided  by  prime 

• (EX)  Radiation 

• (EX)  Dynamics 

• Information  from  other  program  issues 

• Government  SPO  to  SPO 

Contract  Type 

• Fixed  price 

• Best  for  cost  control 

• Technical  scope 

• Cost  vs  technical  drivers 

• Poor  choice  for  development  contract 

• Cost 

• Need  tight  realtime  controls 

• Contract  supportive  of  technical 
excellence 

• Awards/Incentives 

Schedule 

• Tight  but  realistic  schedule 

• Design  time 

• Parts  procurement 

• Breadboards 

• Engineering  model 

• Fabrication 


• Test 

• Definable  milestones 
Preliminary  Design  Review  Period 

• Requirements 

• Integrate  schematics  into  total  system 

• Does  design  do  what  was  intended 

• Performance  changes 

• Fabrication 

• Process  Controls 

• Parts 

• Facilities 

• Test 

• Layout  Test  Plan 

• Sufficient  Test  Points 

• Support  Equipment  Design 

Critical  Design  Review  Period 

• Performance  Requirements 

• Redundancy  (SPF) 

• Specialists 

• Nonperformance  Design  Items 

• Mechanical/Vibration 

• Derating 

• Timing 

• Thermal 

• Material 

• PC  Board  Layout  (Inspectability) 

• Test 

• All  Mission  Sequences  Tested 

• All  Redundancy  Tested 

• 1540  and  1541  Requirements  Properly 
handled 

• Instrumentation  (Testability) 

• Protype  Hardware 

Prototype/Engineering  Model  Hardware 

• Build  To  Production  Drawings 

• Commercial  Parts 

• Production  Fabrication  Personnel 

• Test 

• Performance 

• Environmental 

• Temperature 
« Vacuum 

• Vibration 

• Instrumentation 

• Thermocouples  on  Critical  Boards 

• Accelerometers  on  Critical  Boards 

• High  Response  Current  Measurements 
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Richard  Schwartz 


• Test 

Definable  milestones 


Vice  President  and  Division  Director 
of  Satellite  Systems 
Rockwell  International 


Designing  For  Mission  Assurance 
at  Subcontractor 

Management 
Contract  Type 
Schedule 
PDR  Period 
CDR  Period 

• Prototype  Hardware 
Hardware  Fabrication 
Test 

Shuttle 

Management 

Establish  single  point  for  contract 
management  at  prime 

• Engineering  vs.  contracts 

Brief  total  subcontract  team  on  where  his 
piece  fits  in  the  total  program 

Visits 

• Often  by  key  members  of  management 
Technical  assistance 

t Some  areas  must  be  provided  by  prime 

• (EX)  Radiation 

• (EX)  Dynamics 

Information  from  other  program  issues 

• Government  SPO  to  SPO 

Contract  Type 
Fixed  price 

• Best  for  cost  control 

• Technical  scope 

• Cost  vs  technical  drivers 

• Poor  choice  for  development  contract 
Cost 

• Need  tignt  realtime  controls 

• Contract  supportive  of  technical 
excellence 

Awards/Incentives 

Schedule 

Tight  but  realistic  schedule 

• Design  time 

• Parts  procurement 

• Breadboards 

• Engineering  model 

• Fabrication 


Preliminary  Design  Review  Period 

• Requirements 

• Integrate  schematics  into  total  system 

• Does  design  do  what  was  intended 

• Performance  changes 

• Fabrication 

• Process  Controls 

• Parts 

• Facilities 

• Test 

• Layout  Test  Plan 

• Sufficient  Test  Points 

• Support  Equipment  Design 

Critical  Design  Review  Period 


• Performance  Requirements 

• Redundancy  (SPF) 

• Specialists 

• Nonperformance  Design  Items 

• Mechanical/Vibration 

• Derating 

• Timing 

• Thermal 

• Material 

• PC  Board  Layout  (Inspectability) 

• Test 

• All  Mission  Sequences  Tested 

• All  Redundancy  Tested 

• 1540  and  1541  Requirements  Properly 
handled 

• Instrumentation  (Testability) 

• Protype  Hardware 

Prototype/Engineering  Model  Hardware 

• Build  To  Production  Drawings 

• Commercial  Parts 

• Production  Fabrication  Personnel 

• Test 

• Performance 


• Environmental 


• Temperature 

• Vacuum 

• Vibration 

• Instrumentation 


• Thermocouples  on  Critical  Boards 
t Accelerometers  on  Critical  Boat ds 

• High  Response  Current  Measurements 


Fabrication 


• Process  Controls  (Documented  and  Agreed 
To) 

• Solder 

• Conformal  Coat 

• Component  Mounting 

• Module  Level  Test 

• Functional  at  end  of  fabrication  at 
high  and  low  temperature 

• Fabrication  Audit 

• By  process  disciplined  engineers 

• Paper  System 

• Review  first  production  hardware 

• Discrepancy  Reporting 

• Make  all  failures  reportable 

• Inspection 

Test 

• MIL-STD-1 540A 

• Failure  Reporting/Analysis 

• Report  and  Analyze  ALL 

• Resolve  Prior  to  Acceptance 

• Data  Review 

• Detail  Review  at  All  Levels 

• Module 

• Box 

• Environmental 

• Establish  Buy-Off  Team  for  Each  Box 
Box  Level  Acceptance  Test  Flow  (Per  1540) 

□ Functional 

□ Random  Vibration 
O Functional 

I I Thermal  Cycling  * 17  cycles 
□ Functional 

Thermal  Vacuum  Cycl  ing  I li  Cycle 

Functional O 
Final  Inspections O 
Ship  a 

Shuttle 

• Weight 

• Volume 

• In-Orbit  Test 

Items  For  Added  Emphasis 

• Workmanship  Standards 

t Better/Faster  Flow  of  Other  Program  Issues 
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TEST  CONSIDERATIONS 


I 


Joseph  Statsinger 
Vice  President  and  General  Manager 
The  Aerospace  Corporation 

Test  Program  Objectives 

• Evaluate  Design  Alternatives  as  Design 
Evolves 

• Demonstrate 

• Design  Adequacy 

• Functional 

• Environmental 

• Compatibility  of  System  Elements 

• Overall  System  Performance  to  Meet 
Specifications 

• Provide  assurance  of  proper  manufacture, 
processing  and  workmanship 

• Predict  the  future 
Elements  of  Test  Program 

• Test  Planning 

• Developmental 

• Stress 

• Margin 

• Life 

• Functional 

• Qualification 

• Acceptance 

• Design  of  product  for  testability 

■ Design  and  control  of  test  equipment 

• Design  of  system  for  test  data  processing 
and  evaluation 

• Provision  of  hardware  and  software 
resources  for  failure  analysis  and 
diagnostics 

• Documentation 

• Personnel  training  and  motivation 
Product  Design  for  Testability 

• Availability  and  accessibility  of  test 
points 

• Accessibility  and  stability  of  control 
settings 

• Deisng  for  realistic  testing 

• Separability  of  redundant  elements  for 
testing 

• Accessibility  of  sub-assemblies  for 
removal  and  replacement 

• Availability  of  telemetry  points 
Design  of  Test  Equipment 

• Provide  for  required  accuracy  and 
resolution 

• Provide  capability  for  positive  and 
negative  testing 

• Design  for  fault  protection 


• Provide  interlocks 

• Consider  human  engineering  aspects 

• Assure  adequate  design  controls 

• Specifications 

• Standards 

• Configuration  Management 
Predicting  the  Future 


• Short  Term  - Expendable  Boosters 

• Stress  Testing 

• Test  Realism 

• Timing  of  Tests 

• Long  Term  - Spacecraft 

• Real  Time  Life  Testing 

• Accelerated  or  Simulated  Life  Testing 

• Stress  Testing 

• Modeling  and  Phenomenology 

• Test  Real  ism 


Software  Categories 


• Mission  Critical  - Time  Urgent 

• Software  anomalies  produce  catastrophic 
failure 

• Mission  Critical  - Not  Time  Urgent 

• Recovery  Procedures  Available 

• Off  Line  - Not  Mission  Critical 


Software  Testing  Considerations 

• Mission  Critical  - Time  Urgent 

• Provide  automated  redundant/back-up 
capability  if  possible 

• Test  all  modes  and  mode  combinations 

• Provide  for  independent  test  and 
validation 

• Not  Time  Urgent 

• Test  all  modes  and  mode  combinations 
if  possible 

• Validate  safing  procedures 

• Validate  recovery  procedures 

• Provide  capability  for  and  validate 
on-board  re-programming  for  spaceborne 

software 

Summary 

• Some  areas  needing  further  study 

• Development  Testing 

• Product  Design  for  Testability 

• Availability  of  Test  Beds 

• Design  and  Control  of  Test  Equipment 

• Testing  for  Long  Term  Performance 
Prediction 

• Software  Testing  and  Validation 

• Personnel  Training  and  Motivation 
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CONCLUSIONS 


Paul  E.  Wright 
Vice  President,  Engineering 
RCA  Corp. 

Now  that  you  have  heard  from  five  of  the  members  of 
the  Panel,  let  me  conclude  with  a few  observations. 
There  is  a popular  myth  that  most  of  the  failures 
on  spacecraft  are  mechanical,  that  derives  probably 
from  the  spectacular  nature  of  a failure  in  the 
launch  process.  Statistics  that  I have  studied 
indicate  that  spacecraft  problems  serious  enough  to 
affect  mission  objectives  divide  about  equally 
between  mechanical  and  electrical  failures.  The 
general  approach  to  be  taken  in  reducing  mechanical 
failures  is  substantial ly  different  than  that  for 
protection  against  electrical  failures.  Careful 
attention  to  concept,  extensive  modeling,  and  care- 
ful subassemblyv'and  testing,  are  approaches  to 
mechanical  problems.  Most  electrical  failures  can 
be  dealt  with,  however,  through  process  standard- 
ization and  part  qualification. 

It  is  clear  to  everyone  in  this  audience,  I am  sure, 
that  the  Shuttle  era  will  change  the  boundary  con- 
ditions against  which  we  design  spacecraft.  We 
will  see  substantial  reductions  in  the  number  of 
launch  failures  because  of  a well  tested,  well 
designed.  Shuttle  launch  system.  We  will,  however, 
have  to  deal  with  the  new  questions  of  reliability 
that  come  from  deploying  the  Shuttle  cargo.  A new 
set  of  operations,  involving  mechanical  and  stabil- 
ization procedures,  will  provide  their  own  set  of 
reliability  considerations.  Designers  must  also 
keep  in  mind  the  new  opportunities  provided  by 
Shuttle  to  do  new  things,  like  the  repair  of  a 
spacecraft,  the  adjustment  of  its  orbit,  and  even 
the  retrieval  of  a temporarily  defective  spacecraft 
for  subsequent  repair  and  redeployment.  Shuttle 
offers  an  entirely  new  set  of  cost  tradeoffs.  Now, 
the  system  is  essentially  volume  sensitive  rather 
than  weight  and  power  sensitive  as  has  been  our 
past  experience.  I could  go  on  for  an  hour  but  let 
me  terminate  the  comments  on  the  Shuttle  era  by 
pointing  out  that  we  will  see  a multiple  interface 
situation  with  Shuttle;  the  government  will  be  pay- 
ing many  industrial  firms  to  build  satellites,  all 
of  which  will  come  together  in  the  bay  of  a single 
Shuttle  spacecraft  for  transport  to  suborbit.  This 
is  a systems  integration  problem,  as  challenging  as 
those  faced  in  the  Apollo  p-ogram. 

My  final  concluding  remarks  will  deal  with  some 
future  trends.  First,  the  role  of  "non  industry1' 
org;.nizations  like  JPL,  Lincoln  Laboratory,  and 
Aerospace  in  the  design  process  will  be  an  expand- 
ing one.  Secondly,  the  Space  Shuttle  offers  an 
opportunity  for  future  spacecraft  design  which  will 
have  far  reaching  impact;  that  is,  assembly  of 
structures  in  space.  This  will  change  our  design 
philosophy  in  areas  such  as  subassembly  size,  level 
of  test,  repair  and  replacement.  A third  consider- 
ation for  future  design  trends  is  appropriate  use  of 
the  data  base.  In  spite  of  the  fact  that  we  have 
all  been  in  this  business  for  well  over  a dozen 
years,  it  is  still  a very  young  industry.  We  are 
just  now  beginning  to  build  the  data  base  so  that 
statistically  significant  conclusions  can  be  drawn 
and  used  for  the  basis  for  generalized  design  rules. 
Finally,  I believe  that  the  aerospace  industry, 

(both  the  government  and  industrial  parts)  must 
reassess  its  conservative  attitudes,  particularly 
with  regard  to  advances  in  electronics.  We  can  not 
afford  to  become  extensive  risk  takers  in  the 


decades  of  the  80' s and  90' s but  we  should 
re-examine  the  leader/follower  roles.  Our  practice 
has  been  to  use  technology  in  spacecraft  only  after 
it  has  been  thoroughly  exercised  by  other  non-space 
users.  The  result  has  been  a substantial  lag  by 
the  aerospace  industry  in  the  use  of  solid  state 
products.  I suggest  that  Mission  Assurance  may 
very  well  be  reduced  rather  than  enhanced  by  that 
attitude.  I think  the  future  demands  that  we  con- 
sider optimizing  our  life  cycle  costs  by  accelerat- 
ing the  maturity  process  of  LSI  and  VLSI,  so  that 
we  can  experience  the  significant  mission  assurance 
advantages  that  are  inherent  in  these  technology 
'concepts.  The  use  of  distributed  microprocessors, 
solid  state  memories,  solid  state  replacements  for 
traveling  wave  tube  amplifiers,  etc.,  will  be  the 
basis,  I believe,  for  improvements  in  Mission 
Assurance  through  design  for  the  decade  ahead. 
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0830-0845 

Introduction 

Mr.  Edward  T.  Bobak,  Aerospace 

0845-0915 

Overview 

Mr.  Emery  1.  Reeves,  TRW 

0915-0945 

"Program  and  System  Design 
Approaches" 

Mr.  Anthony  J.  lorillo,  Hughes  Aircraft 

0945-1015 

"Product  Design  and  FMEA" 

Mr.  Phillip  D.  Crill,  Ford  Aerospace  & 
Comm.  Corp. 
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Break 

1030-1100 

"Design  for  Test" 

Mr.  Anthony  J.  Aukstikalnis,  RCA 
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1100-1130 

"Designers'  Role  in  Manufacturing, 
Test&  Operations" 

Mr.  Robert  C.  Koche,  Lockheed  LMSC 
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"Prevention  of  Design  Faults: 
Redundancy" 

Mr.  Kenneth  P.  Timmons, 

Martin  Marietta 
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1330-1500 

Subgroup  Discussions  Aimed  at 
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1515-1700 
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DESIGN  FOR  MISSION  ASSURANCE  WORKSHOP 
-OVERVIEW- 

Mr.  Edward  T.  Bobak 
Systems  Engineering  Director 
The  Aerospace  Corporation 


Imperfections  in  design  are  the  most  predominant 
causes  of  catastrophic  mission  failures.  In  order 
to  alter  the  trend  and  improve  the  probability  of 
mission  success,  it  is  necessary  to  review  the 
design  process  in  its  elemental  forms.  By  so 
doing,  weaknesses  can  be  identified  and  construc- 
tive steps  initiated  to  improve  the  process. 

The  format  of  the  Design  Workshop  was  designed  to 
focus  the  attention  of  participants  on  significant 
elements  of  the  design  process.  First,  by  respond- 
ing to  a detailed  questionnaire  that  requested 
opinions  on  the  importance  of  design  elements. 
Secondly,  by  utilizing  presentations  on  selected 
topics  by  industry  experts.  Finally,  by  group 
discussion  of  agreed-upon  critical  issues  that  lead 
to  constructive  recommendations  for  change. 


THOUGHTS  ON  IMPROVING  THE  DESIGN  AND 
VALIDATION  PROCESSES 

Anthony  J.  Iorillo 

Associate  Manager,  Technology  Division 
Space  and  Communications  Group 
Hughes  Aircraft  Company 


As  this  conference  attests.  Mission  Assurance  is  a 
broad  topic  involving  many  disciplines  and  points  of 
view.  Since  each  of  us  tends  to  be  a product  of 
specific  experience,  the  class  of  failures  or  errors 
to  which  we  are  most  sensitive  varies.  Hopefully, 
the  frank  sharing  of  our  problems  and  thoughts  on 
avoiding  repetition  of  our  errors,  when  summed,  will 
serve  to  inspire  creative  dialogue  and  motivate  the 
upgrading  of  techniques  to  the  benefit  of  all.  Con- 
sistent with  this  spirit,  I will  discuss  facets  of 
our  experience  with  misfortune  at  Hughes  and  share 
with  you  some  ideas  on  how  we  might  improve  our 
performance.  Specifically,  I wish  to  focus  on  the 
management  of  the  design  and  validation  process  for 
satellite  systems. 

Encompassing  one  hundred  and  fifty  satellite  years, 
our  flight  logs  record  a wide  spectrum  of  equipment 
failures  and  degradations.  Among  these  are  found 
the  typical  random  electronic  component  failures 
and  the  premature  degradation  of  batteries, 
receiver  gains,  traveling  wave  tubes,  mechanism 
friction  and  the  like.  Another  significant  class 
of  entries  involves  flaws  in  workmanship  or  margin- 
al unit  designs  which  escape  our  screening  test 
filters.  Generally,  these  failures  have  been 
accommodated  by  redundancy,  initial  margin  or  oper- 
ational flexibility  inherent  in  the  satellite 
design.  Missions  have  degraded  below  expectation 
But  have  been  performed  nonetheless. 

The  failures  which  had  immediate  catastrophic  con- 
sequences involved  no  flaw  in  workmanship,  random 
part  failure  or  component  degradation.  I refer  to 
the  launch  of  vehicles  whose  characteristics  were 
not  well  understood,  for  which  test  and  analyses 
were  inadequate.  Specific  examples  are: 

1.  Explosion  of  Intelsat  II  apogee  kick 
motors  because  we  did  not  properly 
analyze  the  thermal  conditions  at 
ign1'  tion; 

2.  Instability  of  NASA  Application  Test 
Satellite  due  to  late  addition  of  a 
liquid  heat  pipe  whose  effect  on  nutation 
was  incorrectly  assessed;  and 

3.  Loss  of  a portion  of  a payload  on  a USAF 
satellite  because  a single  fuse,  in 
series  with  redundant  receivers,  blew 
during  a switching  sequence. 

In  addition  to  outright  failure,  we  have  had  our 
share  of  close  calls  whose  consequences  were 
serendipitously  tolerable. 

1.  USAF  TACSAT  nutation  resulted  from  an 
oversight  in  the  specification  of  its 
mechanical  despin  assembly  damping 
characteristics. 

2.  Much  larger  than  anticipated,  dynamic 
unbalance  of  a meteorological  satellite 
resulted  from  Incomplete  analysis  of 
vehicle  dynamics. 


3.  Hang  up  of  a number  of  pointing  mecha- 
nisms and  rubbing  between  a despun/spun 
interface  because  we  did  not  properly 
account  for  thermal  environment. 

4.  Unexpected  mode  change  in  attitude  control 
system  on  a USAF  satellite  because  of  the 
design  shared  clock  signals  with  a telem- 
etry unit  which  was  by  intent  automati- 
cally turned  off  in  eclipse. 

This  partial  list  of  examples  serves  to  fix  ideas  on 
the  type  of  errors  I refer  to.  Hindsight  makes  them 
seem  the  result  of  ineptness.  Yet  they  were  made  by 
motivated  and  competent  people  whose  successful 
accomplishments  provide  testimony  to  their  skills. 

I have  reviewed  recently  a summary  of  failures  and 
anomalies  compiled  by  SAMSO.  I find  that  design 
oversight  occurs  often  enough  in  this  larger  data 
base  to  suggest  that  it  is  a general  problem.  I 
also  find  that  our  industry  has  reacted  to  this  body 
of  experience.  Recognizing  the  large  increase  in 
complexity  of  military  satellite  designs  in  the  last 
decade,  normalized  performance  has  in  fact  improved. 
The  prospect,  however,  is  that  the  next  generation 
of  systems  will  be  even  more  complex  and  a fresh 
generation  of  engineers  will  be  involved.  Unless 
we  are  concerned  with  the  passing  forward  of  infor- 
mation and  improved  methods,  we  can  be  assured  that 
old  errors  will  be  repeated  along  with  the  predict- 
able new  set. 

I have  attempted  to  analyze  the  underlying  reasons 
for  the  failures  cited  above  with  the  hope  of 
extracting  some  generalizations.  Notwithstanding 
the  classic  danger  involved  in  this  attempt,  I sub- 
mit the  following  views: 


1.  A shortcoming  in  our  management  system  is 
the  extent  to  which  we  review  systemati- 
cally the  implicit  assumptions  underlying 
analytical  models  used  to  predict  and 
validate  performance  and  establish  quali- 
fication test  conditions  and  criteria. 

We  assume,  too  often,  that  the  specialists 
involved  set  the  problem  up  accurately  and 
we  start  the  review  process  by  accepting 
and  assessing  results.  Few  program  mana- 
gers, system  and  mission  engineers  or 
hardware  designers  are  ever  involved  in  a 
systematic  review  of  an  exposition  of  the 
analytical  model  and  its  degree  of  fidel- 
ity to  actual  hardware  or  the  intended 
environment  and  application.  This  is 
contrasted  with  the  relatively  disciplined 
manner  in  which  we  require  hardware 
designs  to  undergo  rigorous  validation 
tests.  Yet  as  we  experience,  omissions 
in  modelling  and  simulations  dynamics  and 
control,  thermal  and  in  some  fewer  cases 
structural  modelling-are  the  basic  causes 
of  our  most  dramatic  failures. 


2.  As  a body,  program  and  technical 

management  rarely  know  in  detail  what 
elements  and  possible  modes  of  the  design 
are  not  tested  functionally  or  environ- 
mentally. Consequently  we  test  some 
modes  accidentally  for  the  first  time  late 
in  system  test  or,  worse,  in  orbit  and  are 
surprised  by  the  results.  (In  a specific 
case,  failure  detection  and  toggle  cir- 
cuitry which  was  relied  on  to  assure 
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automatic  switchover  to  a redundant  con- 
trol unit  in  the  event  of  failure  was 
never  burned  in  during  system  test.  Fail- 
ure is  not  a standard  sequence.  The  con- 
tract required  a 500  hour  burn-in  of  all 
units  in  system  test.  However,  much  of 
the  circuitry  in  many  units  is  effectively 
dormant  even  with  power  on.)  More  gener- 
ally, many  modes  of  a complex  interactive 
system  which  do  not  contribute  to  primary 
mission  performance  parameters  or  occur 
in  normal  operations  are  frequently  over- 
looked in  synthesizing  the  system  test 
program.  There  is  basis  to  the  lore  that 
we  learn  more  about  the  design  by  inves- 
tigating the  results  of  command  sequence 
errors  in  system  test  than  we  learn  when 
exercising  correct  normal  sequences. 

3.  The  discipline  of  performing  and  reviewing 
FMEAs  is  not  yet  a well  established  part 
of  our  system  engineering  process.  It  is 
tedious,  difficult  and  requires  experience 
to  perform  a comprehensive  analysis.  We 
are  only  now  beginning  to  recognize  this 
and  must  plan  our  resources  in  advance  at 
a level  commensurate  with  the  task. 

Contractor  Review 

We  are  attempting  to  establish  improved  review  pro- 
cedures to  surface,  in  a systematic  manner,  under- 
lying assumptions  in  analytic  models  used  to  estab- 
lish the  adequacy  of  design.  In  particular  we  plan 
to  review  dynamic  and  control  models,  thermal  anal- 
ysis models  and  structural  dynamics  and  stress 
models.  In  the  presence  of  a genera?  senior  audi- 
ence comprising  engineers  familiar  with  mission 
sequences  and  hardware  operations,  as  well  as  tech- 
nical specialists,  we  plan  to  require  analysts  to 
explain  the  implicit  assumptions  in  their  modelling, 
the  simplifications  made  to  render  the  analysis 
tractable  and  the  depth  of  the  data  base  from  which 
they  extract  physical  parameters.  What  we  attempt 
to  achieve  is  a common  understanding  of  the  fidelity 
of  the  modelling  to  the  detailed  design,  the  actual 
environment,  and  the  intended  use  of  the  satellite. 
In  many  cases,  we  find  this  type  review  inspires 
relatively  simple  measures  to  develop  experimental 
data  where  the  models  fall  short.  We  also  hope  to 
uncover  systematically  those  facets  of  the  analysis 
for  which  no  validation  test  is  planned.  The  intent 
of  course  is  to  surface  risks  and  uncertainties  to 
programmatic  levels  which  can  take  correction  or 
alternate  action. 

The  key  to  these  views  is  the  recognition  of  the 
leverage  the  analytical  functions  have  on  mission 
success  and  the  planning  of  resources  in  advance  to 
see  that  the  analyses  are  carried  out  thoroughly 
during  development.  Last  minute  independent  review 
teams  assembled  prior  to  launch  attempting  to  review 
years  of  technical  activity  cannot  be  relied  on  to 
provide  a comprehensive,  in-depth  judgment.  In 
general,  the  problem  data  they  evaluate  are  assem- 
bled for  them  by  program  staff  and  they  rarely 
generate  new  concerns.  A staff  of  competent  engi- 
neers assembled  at  COR,  for  example,  who  are 
chartered  to  accomplish  Independent  validation 
analyses  comparing  notes  with  the  contractor  on  a 
scheduled  basis  is  a more  reliable  investment  of 
Independent  effort.  The  activity  of  th4'  staff  may 
be  viewed  as  an  early  phase  of  the  Independent 
readiness  review  task. 


For  this  approach  to  be  effective  several  condi- 
tions should  be  avoided: 

1.  The  collusion  of  contractor  and  Aerospace 
analysis  teams  in  the  difficult  to  resist 
improvement  of  the  design  process  beyond 
the  acceptable  point  of  adequacy. 

2.  The  unwillingness  of  the  validation  team 
to  accept  sensible  risks  because  valida- 
tion is  by  its  very  nature  a no  win  task. 

3.  The  creation  of  a bureaucratic  entity 
which  outlives  its  usefulness  - the  teams 
should  be  dismantled  after  the  system 
design  is  verified  by  first  article 
performance  in  orbit. 

It  is  my  belief  that  proper  attitude  in  the  USAF, 
Aerospace  and  contractor  management  organizations 
can  preclude  these  potential  hazards. 

The  Future 

I have  not  mentioned  the  root  cause  of  the  design 
and  qualification  problems  I discussed  earlier 
because  it  is  obvious.  Unlike  our  colleagues  in 
airborne  and  ground  based  aerospace  development, 
we  have  not  had  the  luxury  of  flight  test  or  In 
situ  test  for  final  verification.  Lacking  zero-g 
and  sufficiently  large  STV  test  chambers,  we  rely 
more  on  analytical  verification  than  most  technical 
development  fields.  The  reason  is  simply  that 
launch  costs  are  so  high  that  we  have  not  been  able 
to  afford  the  routine  test  of  prototypical  vehicles 
in  orbit.  Consequently,  we  have  had  qualification 
test  vehicles  sitting  in  hangars  while  we  completed 
the  design  process  with  the  first  flight  articles. 

The  advent  of  the  Space  Transportation  System 
should  cause  a dramatic  change  in  this  way  of  doing 
business.  With  proper  management  of  the  STS 
resource,  a large  decrease  in  cost  should  be  forth- 
coming such  that  launch  of  a satellite  will  be  a 
small  fraction  of  satellite  cost.  The  implementa- 
tion of  a test  flight  in  the  prototype  development 
program  should  become  as  routine  as  a final  test 
flight  for  new  aircraft. 

I look  forward  to  this  prospect  because  it  would 
allow  us  to  do  a much  better  job  of  finishing 
development  before  we  commit  large  investments  in 
flight  articles  to  operation. 

I hope  those  thoughts  were  of  general  interest  and 
that  you  find  some  of  the  ideas  useful. 

Similarly  we  are  striving  to  establish  a more 
formal  dialogue  between  mission  operations  special- 
ists, system  engineers,  hardware  designers  and  sys- 
tem test  engineers.  The  normal  channel  of 
information  flow  through  specifications  is  just  not 
sufficient  in  complicated  system  developments  to 
assure  the  hardware  designer  understands  suffi- 
ciently well  the  details  of  the  external  or  the 
interactive  environment,  within  which  his  equipment 
must  function.  We  hope  to  surface  possible  Incom- 
patibilities, interactions  and  modes  which  are  not 
Intended.  Further  we  will  ask  formally  which  part 
of  the  hardware  is  not  exercised  environmentally  or 
functionally  in  the  test  program  at  unit  and  subsys- 
tem levels.  This  latter  activity  will  be  Imple- 
mented In  concert  with  formal  performance 
verification  analyses  which,  though  tailored  to 


54 


contract  end  item  requirements,  comprise  the  basic 
driving  force  for  structuring  the  functional  and 
environmental  test  program. 

Independent  Review 

Early  in  the  SDS  program,  as  a result  of  experience 
with  the  dynamic  instability  encountered  in  the 
OSCS  11  program,  an  independent  Aerospace  stability 
and  control  validation  team  was  established.  This 
Aerospace  team,  comprising  a competent  staff  of 
analysts,  worked  in  parallel  with  Hughes  to  model  in 
detail  and  analyze  the  ambitious  SDS  control  system. 
As  a result  of  the  in-depth  work  of  this  team,  the 
technical  reviews  of  our  progress  in  this  area  were 
thorough  and  constructive,  flaws  in  our  design 
were  uncovered  by  this  Aerospace  team  and  a cooper- 
ative effort  was  undertaken  to  correct  the  defi- 
ciencies. As  the  Hughes  systems  manager,  I was 
always  grateful  for  having  this  resource  available 
to  help  whenever  problems  arose.  Similar,  though 


not  always  as  comprehensive,  or  as  cooperative, 
independent  validation  analyses  were  performed 
during  the  program  in  areas  which  seemed  to  us 
critical.  The  independence  of  the  Aerospace  team 
and  the  often  different  perspective  it  brought  to 
bear  on  the  task  were  important  factors.  I propose 
that  this  independent  validation  be  extended  in  a 
uniform  manner  to  all  areas  of  the  validation  pro- 
cess which  cannot  be  verified  by  test.  The  cost  to 
the  government  would  be  a relatively  small  percent- 
age of  the  developmental  cost  for  any  system.  More- 
over, the  effectiveness  of  Aerospace's  monitoring 
function  would  be  greatly  improved  because  they  will 
become  much  more  familiar  with  all  important  details 
than  is  possible  through  reviewing  contractor  work. 
An  extension  of  this  concept  to  the  task  of  develop- 
ing a model  for  all  possible  satellite  modes  in  the 
manner  I described  earlier  makes  sense.  An  inde- 
pendent construction  of  the  mode  matrix  would  result 
in  a better  review  of  the  design  and  the  contrac- 
tor's qualification  test  proqram. 


Omissions  in  simulation  and  analysis  of  thermal 
environment 

Omissions  in  simulation  and  analysis  of  vehicle 
dynami cs 

Unanticipated  Interaction  of  vehicle  modes 
Single  point  failures  not  screened 
Observations  of  Technical  Management  Shortcomings 
Review  and  validation  of  analytical  models  not 


talnties  not  surfaced  to  program  management 
systematically 


System  qualification  te't  programs  focus  on 
primary  performance  parameters  and  normal 
operations;  interaction  of  nonstandard  modes 
not  systematically  analyzed  or  stimulated 


FMEA  discipline  and  review  not  yet  established 
part  of  system  engineering  function 


Suggestions  For  Improvement  of  Review  Procedures 

Implicit  assumptions,  simplifications,  and 
depth  of  data  bases  involved  in  performance 
verification  analyses  should  be  reviewed  sys- 
tematically by  senior  audience  of  system 
engineers,  mission  operations  engineers,  test 
engineers  and  program  management  as  well  as 
technical  specialists 

Standard  and  nonstandard  satellite  modes 
should  be  systematically  identified;  modes  not 
tested  should  be  surfaced  to  management 

Independent  validation  should  be  strengthened 

Hughes  Satellite  Programs 

Syncom 
Intelsat  II 

Appl ications  Technology 
Satellite  (ATS) 

ATS  (Gravity  Gradient) 

T AC SAT 

Intelsat  IV  and 
Derivatives 


New  Designs  11 
Total  Launches  45 
Booster  Failures  5 
Satellite  Failures  at  Insertion  3 


First  of  a kind  with  significant  6 
design  error 

Total  satellite  years  in  orbit  150 


HS  318 

Satellite  Data  System 
Anik  and  Derivatives 

Marisat 

GMS 


The  Future 


Proper  utilization  of  STS  will  lower  launch 
costs 


Launch  of  prototypes  now  "hangar  queens"  will 
be  affordable 

Qualification  will  include  orbital  tests 
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PRODUCT  DESIGN  AND  FMEA 
by  Phi 1 1 ip  D.  Crill 

Staff  Engineer  to  The  Manager  of  Programs 
Ford  Aerospace  & Comm.  Corp. 


Ed  Bobak  has  asked  me  to  talk  about  the  use  of  FMEA 
in  connection  with  design  for  Mission  Assurance. 
These  words  immediately  turn  me  off.  How  can  I 
assure  a mission?  The  "mission"  for  a military 
communication  satellite  is  to  allow  some  general 
who  is  completely  surrounded  by  Russian  tanks  in 
the  middle  of  Bavaria  to  get  on  his  portable  radio 
and  obtain  permission  from  the  President  to  drop  a 
couple  of  "nukes"  on  the  tank's  headquarters.  I 
can't  assure  that  the  general  will  get  through,  my 
company  can't  assure  that,  and  I doubt  even  SAMSO 
can  assure  that.  What  I can  assure  is  that  when 
the  satellite  gets  into  orbit,  it  will  operate  as 
predicted. 

Operational  people  are  almost  as  smart  as  equipment 
designers  and  suppliers.  They  come  up  with  amazing 
methods  when  necessary  to  utilize  malfunctioning 
equipment  to  accomplish  their  missions.  Notwith- 
standing their  ingenuity  and  mission  success,  we  as 
equipment  suppliers  have  failed  in  our  mission  when 
such  extra  steps  are  required.  Therefore,  I will 
direct  my  remarks  to  one  facet  of  the  job  we  under- 
take to  accomplish,  assurance  that  the  equipment 
will  work  as  predicted. 

We  can  dream  dreams  of  glorious  equipment,  we  can 
plan,  analyze,  or  set  up  elaborate  operational 
schemes,  but  when  the  equipment  goes  into  operation 
it  must  work  reliably.  To  achieve  that,  only  two 
things  really  count;  first,  what  was  put  on  the 
manufacturing  drawing,  and  second,  how  faithfully 
the  manufacturer  followed  the  drawing.  It  is  an 
old  cliche,  but  nonetheless  true,  that  we  can  only 
build  in  quality  and  reliability.  We  can  test  to 
determine  deficiencies  and  inspect  to  determine 
errors  but  in  the  end  operational  reliability  must 
be  built  in.  Anything  we  can  do  to  assure  that  the 
manufacturing  drawings  represent  what  we  intend, 
and  are  the  best  that  can  be  accomplished,  is  a 
large  step  toward  assuring  mission  success.  FMEA 
can  play  a large  part  in  this  vital  step. 

The  purpose  of  FMEA  is,  then,  to  assist  and  let 
me  underline  the  word  assist  the  designer  to  turn 
out  correct  drawings  by  performing  a systematic 
review  of  the  design  with  experts  from  all  disci- 
plines. I underline  the  word  assist  because  FMEA 
can  easily  become  a contest  between  two  equally 
good,  but  different,  ways  of  doing  thinqs.  Also, 
FMEA  can  easily  become  an  entity  unto  Itself.  It 
can  become  an  empire  primarily  motivated  toward 
survival,  more  concerned  with  methodology,  form, 
and  public  posture  than  changes  on  a drawing.  It 
does  no  good  for  someone  to  show  you  a well  docu- 
mented, beautifully  written  and  published  report 
and  then  say  "We  told  you  so"  after  a failure 
occurs  in  orbit.  What  was  put  on  the  drawing  is  all 
that  matters.  Only  the  designer  changes  drawings, 
so  FMEA's  purpose  is  to  assist  the  designer. 

A secondary  output,  of  course,  is  the  check  and 
double  check  which  assures  the  design  Is  sound  and 
potentially  successful. 

To  prevent  a continuous  difference  of  opinion 
between  designer  and  the  FMEA  reviewer,  a well 


defined  and  established  level  of  design  require- 
ments and  design  margins  must  be  maintained.  This 
must  be  adhered  to  by  in-house  designers,  managers 
and  FMEA  reviewers,  as  well  at  by  the  customer,  or 
complete  chaos  reigns  and  the  program  managers  soon 
assume  a referees'  role. 

We  at  Ford  are  proud  of  what  we  consider  to  be  a 
very  low  on-orbit  failure  rate.  Part  of  this  suc- 
cess has  to  be  shared  with  the  use  of  FMEA  and  the 
removal  of  potential  failures  before  they  have  a 
chance  to  occur.  We  have  split  FMEA  into  two  parts 
and  employ  knowledgeable  people  in  both  parts. 

FMEA  is  split  between  electrical  circuit  FMEA  and 
product  design  FMEA. 

Over  the  years  we  have  tried  various  methods  for 
circuit  FMEA.  Fault  tree  analysis  was  used  for  a 
time.  In  this  method,  a logic  diagram  was  con- 
structed of  'what  if 's’,  i.e.,  what  if  this  tran- 
sistor shorts',  and  then  we  worked  upward  to  conclude 
the  transmitter  had  no  output.  This  method  had  a 
lot  of  appeal.  It  was  visible  and  could  be  used  at 
all  levels,  but  it  was  cumbersome  and  wasn't  always 
systematic.  We  also  tried  the  tabular  narrative 
method.  This  method  permitted  more  detail  and  was 
easier  to  follow  for  the  first  few  steps,  but  it  was 
difficult  to  follow  the  effects  of  a part  failure 
all  the  way  to  the  satellite  level.  We  have  now 
adopted  the  MATRIX  METHOD. 

In  the  MATRIX  METHOD,  a series  of  cross-hatched 
matrices  are  formed  where  the  vertical  lines  repre- 
sent constituent  components,  and  horizontal  lines 
represent  effects.  The  intersection  is  coded  with 
possible  failure  modes.  Thus,  for  this  simple  logic 
circuit,  piece  parts  resistors  R-l  through  R-4  and 
the  transistor  Q 1 and  amplifier  U-4  are  repre- 
sented as  vertical  lines.  Likewise,  the  DC  input 
and  return  lines,  the  signal  input  lines  and  the 
signal  output  lines  are  represented  vertically. 

Each  of  these  components  is  allowed  to  fail  in  all 
reasonably  possible  ways.  Transistors,  for  instance, 
fail  short  and  open.  Resistors,  on  the  other  hand, 
would  reasonably  only  fail  open. 

The  effect  of  these  failures  are  then  identified  on 
the  HORIZONTAL  lines.  Thus,  in  our  example,  opening 
or  shorting  of  transistor  Q1  has  the  effect  of  no 
output  on  both  A and  B,  while  the  opening  or  short- 
ing of  U-l  affects  only  output  A.  No  effect  or  out- 
of-spec  effect  is  two  lines.  We  always  include  the 
matrix  although  in  this  particular  example  only 
"no  failures"  result.  A similar  matrix  is  made  up 
for  each  small  circuit  element  in  the  system. 

At  the  next  level,  horizontal  lines  from  Level  1 
become  the  vertical  lines  of  Level  two,  and  added  to 
them  are  other  circuit  matrices.  Using  the  effects 
from  Level  1,  the  effects  at  Level  two  are  identi- 
fied on  the  horizontal  lines.  The  horizontal  lines 
from  this  level  become  the  vertical  for  the  next 
level,  on  up  to  the  final  system  level. 

An  example  of  the  tiered  effect  and  of  how  a failure 
mode  can  easily  be  overlooked  is  the  value  driver 
circuit  for  the  hydrazine  thruster  on  the  NATO  III 
spacecraft.  The  valve  drivers  are  contained  in 
their  own  box  located  on  the  equipment  platform. 

They  are  connected  to  the  thruster  valves  by  cable 
harness.  The  thrusters  are  located  on  the  periphery 
of  the  spacecraft.  They  are  100  redundant  and 
isolated  from  each  other.  Each  thruster  has  its  own 


valve  driver  which  is  isolated  from  all  the  others. 
Not  only  that,  but  within  each  valve  driver  is  parts 
redundancy.  Power  to  operate  the  thruster  valve 
comes  from  a protected  plus  bus  through  the  valve 
coil  to  two  transistors  in  series.  If  one  transis- 
tor fails  short,  the  other  transistor  will  open  the 
circuit  and  close  the  valve.  This  represents  super 
redundancy  and  we  are  theoretically  protected  all 
around.  But  that  is  not  what  the  FMEA  said.  FMEA 
said  one  of  the  failure  modes  is  a short  to  struc- 
ture of  the  harness  wires.  The  power  return  at  one 
point  is  connected  to  the  structure.  Should  a short 
occur  between  thruster  and  driver,  the  valve  would 
be  held  permanently  open  and  the  satellite  would  be 
moved  considerably  out  of  position  and  soon  out  of 
hydrazine.  The  solution  was  fairly  simple.  We 
moved  the  valve  drive  to  the  other  side  of  the 
valve  coil.  That  is,  put  it  between  the  plus  source 
and  the  valve.  Now  should  a harness  short  occur,  no 
current  flows  unless  the  valve  driver  is  turned  on. 
If  the  short  is  hard  enough,  the  thruster  may  not 
respond  when  the  valve  driver  is  activated  and, 
indeed,  the  power  line  fuse  may  blow.  But  the 
spacecraft  is  intact  and  the  redundant  thruster  can 
be  used.  This  particular  circuit  also  had  a very 
serious  and  common  product  failure  mode  which  I will 
discuss  in  a minute.  The  amplifier  consists  of  two 
redundant  amplifier  chains  of  three  series  transis- 
tors each.  The  first  two  amplifiers  are  one-half 
of  a dual  transistor  package.  One  of  the  failure 
modes  is  for  one  of  the  transistors  to  short. 
Unfortunately,  when  one  transistor  shorts,  it  over- 
heats the  second  transistor  and,  in  about  90°'  of 
the  cases,  will  cause  it  to  short  also.  Both  tran- 
sistors shorted  will  cause  the  thruster  valve  to 
remain  open  with  disastrous  results.  Looking  at 
this  circuit  now,  it  appears  obvious  that  the 
design  is  unacceptable  and  that  any  designer  worth 
his  salt  would  not  use  it.  However,  if  it  is  so 
obvious,  why  do  we  have  eight  spacecraft  in  orbit 
right  now  using  this  circuit,  and  why  did  we  launch 
two  of  them  after  the  potential  problem  was  uncov- 
ered on  NATO  III?  FMEA  is  a waste  of  money  unless 
we  believe  the  results  and  do  something  with  them. 

We  at  Ford  believe  the  matrix  system  has  consider- 
able merit  over  other  systems  and  we  are  involved 
in  an  effort  to  computerize  it  to  speed  up  the 
process. 

Product  FMEA  is  not  as  far  advanced  as  circuit  FMEA 
but  has  been  used  at  Ford  on  the  NATO  III  program 
and  on  all  our  commercial  programs  since  NATO  III. 

In  the  valve  driver  circuit  I pointed  out  an  exam- 
ple of  a single  transistor  package  containing  two 
transistors  that  represents  a potential  loss  of 
redundancy.  Another  very  common  area  is  the  printed 
circuit  mother  board  in  a box  that  contains  the 
redundant  circuits.  A failure  in  one-half  the  cir- 
cuits can  easily  propagate  to  the  other  half  To 
circumvent  this,  we  at  Ford  make  a practice  of 
putting  redundant  circuits  in  completely  separate 
boxes. 

Tolerances  and  tolerance  buildup  can  appear  in  some 
very  unexpected  ways.  We  are  familiar  with  clear- 
ances on  moving  parts  such  as  the  despin  motors  or 
the  solar  array  drives,  and  sloppy  mounting  holes, 
where  the  washer  half  falls  through.  Once  we  had  a 
series  of  circuit  failures  attributed  to  ceramic 
chip  capacitors.  Investigation  showed  that  some  of 
the  capacitors  had  small  bumps  on  them  that  stuck 


out  past  the  nominal  limit.  There  was  not  enough 
room  to  fit  them  in,  so  one  of  the  technicians  was 
grinding  off  the  bumps.  The  technician  should  not 
have  done  this,  and  caught  hell  for  it,  but  the  real 
culprit  was  the  designer  who  did  not  look  at  the 
tolerances  adequately. 

Thermal  is  another  especially  nagging  problem.  The 
spacecraft  thermal  personnel  are  charged  with  main- 
taining the  baseplate  of  the  boxes  within  a certain 
temperature  limit.  This  aspect  gets  a lot  of  atten- 
tion and  is  usually  handled  well,  but  down  in  a 
module  there  is  a transistor  that  dissipates  a 
quarter  of  a watt.  That  is  a very  small  amount  of 
power  and  the  transistor  is  operating  well  within 
its  rating.  But  what  about  the  component  that  is 
jammed  in  tight  against  the  transistor  and  encapsu- 
lated in  foam?  Unfortunately,  the  designer  is 
engrossed  by  how  he  is  going  to  jam  all  those  parts 
into  the  small  space  the  spacecraft  designer  allo- 
cates, and  thus  doesn't  get  to  the  subtleties  of  the 
thermal  aspects. 

The  specifications  given  to  the  designer  have  pages 
and  pages  of  requirements  about  space  environment. 
They  cover  all  the  shake,  rattle,  and  roll,  plus 
radiation,  vacuum,  sun  loading,  particle  bombard- 
ment, and  now  even  include  space  charging.  All  of 
this  is  followed  by  a simple  little  paragraph  that 
says  "the  unit  shall  be  designed  to  withstand  the 
factory-to-launch  environment."  The  designer  of  the 
unit  doesn't  have  time  to  know,  and  usually  doesn't 
care,  what  happens  to  his  box  once  it  gets  out  of 
his  shop.  It  is  important,  therefore,  that  someone 
who  does  know,  and  who  does  care,  reviews  the 
design. 

Every  circuit,  every  module,  every  component  of  the 
satellite  must  be  carefully  reviewed  by  someone  con- 
cerned about  thermal , someone  concerned  about 
stress,  someone  concerned  about  creep  and  deforma- 
tion, someone  concerned  about  manufactureability, 
someone  concerned  about  assembly  and  handling. 

Seldom  are  all  these  concerns  embodied  in  one  or 
two  people.  Working  meetings  are  therefore  required 
for  each  step  of  the  process. 

Let  me  conclude  by  saying  that  FMEA  can  be  effec- 
tive in  improving  designs  and  can  also  be  cost 
effective.  On  the  NATO  III  Program,  FMEA  disclosed 
118  single-point  failure  modes.  This  occurred  after 
the  designers  ha'  completed  the  first  round  design, 
and  in  spite  of  a firmly  established  ground  rule 
that  there  would  be  no  single-point  failure  modes. 

Of  this  118,  43  were  correctable  and  were  elimi- 
nated, but  the  others  couldn't  be  eliminated. 

Some  of  the  single-point  failures  were  obvious; 
structure,  AKM,  hydrazine  fuel,  etc.  The  majority, 
however,  were  associated  with  the  communications 
subsystem;  input  and  output  filters,  antennas,  the 
rotary  joint,  and,  worst  of  all,  the  switches  used 
to  switch  the  redundant  active  units  into  and  out 
of  the  circuits. 


A firm  ground  rule  of  "no  single  point  failures"  is 
obviously  unrealistic.  Some  will  exist.  FMEA  can 
identify  all  that  remain  and  allow  a judgment  to  be 
made  as  to  the  seriousness  and  cost  effectiveness 
of  eliminating  them. 

Properly  directed  FMEA  can  be  effective.  As  an 
empire  unto  Itself,  it  can  emasculate  your  profits. 
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DESIGN  FOR  MISSION  ASSURANCE  WITH 
FAILURE  MODE  EFFECTS  ANALYSIS  (FMEA) 


DESIGN  FOR  MISSION  ASSURANCE  WITH 
FAILURE  MODE  EFFECTS  ANALYSIS  (FMEA) 


What  Mission? 

• "Mission",  'ike  "system",  can  mean 
anything. 

• Mission  assurance  can,  of  course,  relate 
to  various  levels  of  desired  performance. 

• Our  mission  as  hardware  designers  and 
builders  is  to  provide  equipment  that 
works,  when  needed,  and  in  a manner  that 
was  predicted. 

What  f'unts  In  Design? 

Only  Two  Things  Really  Count: 

• What  the  designer  puts  on  the  drawing. 

• How  faithfully  the  production  person 
implements  the  drawing. 


~cuit  FMEA  Methodology 
i „jlt  Tree  Analysis 


Pictorial  method  of  projecting  effects  to 
system  level 


Tabular  Narrative  Method 


Contains  valuable  detailed  failure  mode 
and  local  effect  information,  but  is 
difficult  to  interpret  system  impact 


Matrix  Method 


• Systematic  method  applicable  from  lowest 
level  module  to  complete  system 


All  the  rest  (engineering,  test,  QA,  reliabil- 
ity, management)  is  to  assure  that  these  two 
happen  and  happen  correctly. 


FIRST  LEVEL  MATRIX 


Failure  Modes  and  Effects  Analysis  (FMEA) 


• Purpose  is  to  assist  the  designer  to  get 
the  drawings  right  by  doing  a systematic 
review  of  the  design  with  experts  from 
all  disciplines. 

• Secondary  output  is  documented  assurance 
to  management  that  a particular  design 
has  high  probability  of  success. 

• A prerequisite  is  a well  established 
design  and  margin  criterion. 


FMEA  Consists  of  Two  Parts 


Circuit  design  FMEA 
Product  design  FMEA 


SECOND  LEVEL  MATRIX 


Advantages  of  Matrix  Method 

• Identifies  reliability  problem  areas  and 
single  point  failure  modes 

• Systematic  accountability  of  all  design 
elements  including  solder  joints  and  con- 
nector pins 

• Relationship  and  traceability  from  lowest 
to  highest  level 

• Can  be  applied  to  any  type  of  design 
technology 

t Tool  for  effective  audit,  review,  and 
incorporation  of  subcontractor  FMEAs 

• Can  be  used  to  analyze  proprietary  design 
without  design  disclosure 

• Analytical  baseline  can  be  established 
with  design  change  update 

• Electrical  and  product  design  failure 
effects/relationship  can  be  established 


Product  FMEA  Methodology 

t Not  as  well  structured  but  just  as 
necessary 

• Mechanical  equivalent  to  electrical  FMEA 

• Requires  expertise  in  mechanical  engi- 
neering, thermal  design,  materials 
properties,  and  an  understanding  of 
electri cal -mechani cal  i nterfaces 

Product  FMEA  Targets 

• Single  point  failure  modes  associated  with 
wiring,  circuit  traces,  connector  pins, 
and  solder  joints 

• Single  point  failure  modes  caused  by 
single  parts  having  multi-application 
elements 

• Thermal,  mechanical,  and  electrical 
adequacy  in  locating,  positioning,  and 
fastening  parts 

• Use  of  authorized  parts,  materials,  and 
processes 

• Adequate  sizing,  continuity,  vxj  effec- 
tiveness of  grounding 

• Tolerances  and  tolerance  build-up  in 
mechanical  parts 

• Provisions  to  prevent  degradation  of 
materials  prone  to  cold  flow 

• Adequacy  of  hermetic  seals  and  adequacy 
of  venting 

t Dissimilar  metals  and  moisture  traps 

• Adequacy  of  designs  to  withstand  the 
environments,  on  earth  as  well  as  in 
space 

• Protection  against  propagation  of  failures 
to  redundant  assemblies 

FMEA  Effectiveness 

• On  the  NATO  III  program,  FMEA  identified 
118  single  point  failure  modes  after  the 
designers  had  done  their  thing. 

• 43  of  the  single  point  failure  modes 
were  eliminated. 
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BUILT-IN  TEST  FEATURES 
FOR 

ACHIEVING  MISSION  ASSURANCE 


A generalized  aerospace  system  is  shown  in  Figure  2. 
There  are  usually  four  major  categories  of  input/ 
output  signals: 


by  Anthony  J.  Aukstikalnis  0 Commands  to  configure  the  system  for 

RCA  Astro  Electronics  its  various  modes  of  operation. 

Princeton,  New  Jersey 

• Input  data  which  is  either  transduced  or 
used  by  the  system  internally. 

• Telemetry  data  used  to  communicate  inter- 

Extensive  testing  is  required  to  insure  that  an  nal  system  status.  Both  hardware  and 

aerospace  system  will  meet  or  exceed  performance  software  derived  telemetry  are  included, 

and  lifetime  requirements.  This  paper  explores 

built-in  system,  hardware,  and  software  design  fea-  • Output  data  which  is  either  transduced 

tures  which  give  greater  assurance  that  an  aerospace  input  data  or  data  which  is  generated 

system  is  completely  tested  and  meets  performance  within  the  system  itself, 

requirements.  It  is  intended  to  stimulate  thought 

leading  to  more  extensive  use  of  built-in  test  The  system  under  test  is  characterized  by  limited 

features  for  aerospace  systems.  internal  visibility  and,  consequently,  presents  a 

difficult  test  and  troubleshooting  environment. 

Figure  1 shows  the  hierarchy  of  test  for  an  aero-  Furthermore,  real  time  constraints  place  further 

space  system  and  the  major  test  objectives  at  each  stringent  demands  on  the  test  system.  To  aid  the 

hierarchal  level.  All  testing  is  oriented  toward  test  process,  three  major  categories  of  test  inputs 

weeding  out  design,  component,  manufacturing,  and  and  outputs  are  usually  implemented, 

interface  problems  at  the  lowest  possible  level. 

In  general,  eliminating  problems  at  the  lowest  • Test  stimuli  for  on  board  subsystems  which 

possible  level  where  they  can  be  more  easily  cannot  be  internally  stimulated.  Exam- 

detected  and  solved,  is  quicker  and  cheaper  than  pies  are  solar  array  simulators  for  space- 

trying  to  eliminate  them  during  a subsequent  level  craft  and  electrical  simulators  for  sen- 

of  testing.  This  paper  will  concentrate  primarily  sors  which  may  not  be  able  to  be  realisti- 

on  system  test,  the  highest  hierarchal  level.  cally  simulated  in  any  other  way. 
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TEST  ENVIRONMENT  IS  CHARACTERIZED  BY: 

t REAL  TIME  PROCESSING  CONSTRAINTS 

• LITTLE  INTERNAL  VISIBILITY 

• DIFFICULTY  IN  TROUBLESHOOTING  PROBLEMS 

Figure  2.  Generalized  Aerospace  System 


• Test  points  which  augment  telemetry  data. 

a Instrumentation  usually  for  determining 
system  environmental  (temperature,  shock, 
acceleration)  conditions. 

Table  1 presents  some  system  level  test  considera- 
tions. In  order  to  effectively  test  a complex 
aerospace  system,  the  test  philosophy  and  methods 
must  be  factored  in  during  the  system  design  phase. 
Also,  with  the  advanced  types  of  data  processing 
equipment  available  today,  the  test  function  should 
be  automated  as  much  as  possible  to  all  levels  of 
testing.  This  will  result  in  less  operator  error, 
repeatability,  consistency,  and  more  than  adequate 
historical  records.  Automated  test  equipment  can 
be  programmed  to  make  most  pass/fail  decisions 
while  presenting  to  data  evaluators  only  the 
reports  they  require  for  further  visibility.  It  is 
essential,  however,  that  the  test  equipment  output 
work  to  facilitate  the  data  evaluation  task  and  not 
complicate  it  by  presenting  masses  of  redundant 
data  for  evaluation.  There  should  also  be  inde- 
pendent means  of  verifying  major  system  interfaces 
to  Isolate  problems  to  the  test  system  or  the  aero- 
space system  under  test. 


Table  1.  System  Level  Test  Considerations 
a Factor  test  in  during  system  design 

• Detect  and  correct  problems  at  the  lowest  pos- 
sible level 

• Automate  as  much  as  possible  (at  all  levels) 

• To  minimize  operator  error 

• For  repeatability  and  consistency 

e To  automate  data  reduction  and  presentation 

• For  archival  purposes  (tapes,  printouts) 

• Make  the  equipment  work  for  you  - avoid 
using  it  as  a printer 

• Independently  verify  system  interfaces 

• Independent  command  transmission  check 


• Independent  backup  telemetry  decom/display 
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Table  2 shows  system  design  features  which  facili- 
tate testing  while  giving  more  visibility,  better 
troubleshooting  capability  and  more  comprehensive 
testing.  These  system  design  features  can  be 
divided  into  two  major  categories: 


• Features  which  decrease  system  capa- 
bility, e.g.,  simplified  command  and 
telemetry  formats.  From  a test  view- 
point, simplifying  these  interfaces  leads 
to  less  and  faster  processing  in  what  is 
normally  a very  constrained  real  time 
environment.  Also,  more  comprehensive 
testing  results  since  there  are  fewer 
formats  to  test  and  those  that  are 
present  can  be  tested  more  often.  On  a 
component  count  basis,  the  system  is  also 
more  reliable  since  less  components  are 


required  to  implement  the  simpler  com- 
mand and  telemetry  subsystems. 

• Features  which  increase  visibility  and 
troubleshooting  capability,  e.g.,  test 
points  and  error  codes.  Although  more 
processing  is  required,  the  additional 
testing  and  visibility  gives  more  assur- 
ance that  the  system  will  meet  its  per- 
formance requirements  over  the  life  of 
the  mission. 

Here  we  see  a fundamental  tradeoff  which  accompanies 
the  designing  in  of  built-in  test  features. 

Built-in  test  features  increase  component  count 
and,  therefore,  slightly  decrease  reliability 
estimates  based  on  component  failure  rates.  The 
assurance  gained  by  additional  visibility  at  the 
system  level  far  outweighs  the  lower  reliability 
even  though  it  cannot  be  expressed  mathematically 
as  part  of  a probability  of  success  computation. 


Table  2.  System  Design  Considerations 


Test  Oriented 

Consideration 

Examples 

Effect  on  Test 

Effect  on  System 

Command  formats 

• 

Standardize  on  a few, 

• 

Less  processing 

f 

Less 

flexible 

simple  formats 

• 

Faster  processing 

• 

More 

rel iable 

• 

More 

comprehensive  test 

Telemetry  formats/modes 

• 

Avoid  data  packing 

• 

Less  processing 

§ 

Less 

efficient 

• 

Avoid  irregular  data 

t 

Faster  processing 

• 

Less 

flexible 

organization 

t 

More 

rel  iable 

• 

Avoid  seldom  used 
modes 

• 

More 

comprehensive  test 

• 

Maximize  commonality 
between  modes 

Test  points 

• 

For  higher  frequency 

• 

More  processing 

• 

Less 

reliable 

signals 

• 

More  visibility 

• 

More 

comprehensive  test 

• 

For  telemetry  sub- 
system overflow 

• 

Better  trouble- 
shooting capability 

• 

For  inputs/outputs 
to  external  stimuli 

Error  codes 

• 

For  command  format 

t 

More  processing 

• 

Less 

reliable  unless 

• 

faults 

For  computer  status 
faults 

• 

t 

More  visibility 

Better  trouble- 

used  as  a basis  for 
fault  correction 

shooting  capability 

• 

For  detected/ 
corrected  data  errors 
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Tables  3 and  4 show  specific  hardware  and  software 
design  features  which  increase  a system's  testa- 
bility. Again,  these  design  features  can  be  divided 
into  two  major  categories: 

• Features  which  impose  known  inputs  or 

conditions  on  the  system  operation,  e.g., 
pattern  generators,  calibration  points, 
redundancy  circuit  stimuli,  computer  self 
test  and  software  modeling.  These  fea- 
tures give  better  visibility  and  facili- 
tate troubleshooting  due  to  the  repeat- 
ability and  predictability  of  the  results. 
They,  however,  only  augment  the  testing 


and  should  not  be  used  as  a substitute 
for  testing  the  system  with  actual  data  or 
conditions.  Data  dependent  faults  may 
not  be  detected  without  the  use  of  actual 
data  or  conditions.  Again,  the  tradeoff 
between  slightly  decreased  reliability 
and  increased  assurance  is  evident. 

Features  which  increase  visibility  or 
troubleshooting  capability,  e.g.,  test 
points  and  modifiable  telemetry  formats 
These  features  are  most  valuable  as 
troubleshooting  aids  to  obtain  an  in  situ 
assessment  of  a potential  problem. 


Table  3.  Hardware  Design  Considerations 


Test  Oriented 

Consideration 

Examples 

Effect  on  Test 

Effect  on  System 

Pattern  generators 

• 

BER  checks 

• 

Predictabi  1 ity 

• 

Less 

reliable 

t 

Data  input  simulators 

• 

Easier  unit/ 
subsystem  test 

t 

More 

comprehensive 

test 

• 

Better  trouble- 
shooting capability 

Calibration  points 

• 

A/D  converter 

• 

Predictabi 1 i ty 

• 

Real 

time  accuracy 

ral ibration 

• 

More  visibility 

monitoring 

• 

Radiometer  calibration 

• 

Easier  trouble- 
shooting capability 

Redundancy  circuit 

• 

Commandable  redun- 

• 

More  visibility 

• 

Less 

reliable 

stimuli  (fault 
simulation) 

dancy  switching 

t 

Better  trouble- 

• 

More 

comprehensive 

test 

• 

Data  error  detection/ 
correction 

shooting  capability 

t 

Initiation  of  fault 
tolerant  features 

Test  points 

t 

Test  signal  inputs 

• 

Easier  unit/ 

• 

Less 

reliable 

• 

Test  signal  outputs 

subsystem  test 

• 

More 

comprehensive 

test 

• 

Better  trouble- 
shooting capability 

Modifiable  telemetry 

t 

ROM  for  primary 

• 

Better  trouble- 

• 

Less 

rel iable 

data 

format;  RAM  for 
secondary,  modifiable 
formats 

Shooting  capability 

• 

More 

flexible 

Table  4.  Software  Design  Considerations 


Test  Oriented 
Consideration 

Examples 

Effect  on  Test 

Effect  on  System 

Computer  self  test 

• Memory  checks 

t Instruction  execution 
tests 

t Addressing  mode  tests 

• More  visibility 

• Less  reliable 

• More  comprehensive  test 

Software  modeling 

• Vehic’e  dynamics 

• More  processing 

• Less  reliable 

simulations 

• Sensor  input 
simulations 

• Repeatability/ 
predictability 

• More  comprehensive  test 

Modifiable  telemetry 

• Table  driven  software 

• Better  trouble- 

• Less  reliable 

data 

telemetry  system 

shooting  capability 

• More  flexible 
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Table  5 summarizes  the  main  issues  of  this  paper. 
Built-in  test  features,  if  considered  early  in  the 
system  design  phase,  can  greatly  increase  assurance 
that  a system  will  meet  its  performance  and  life- 
time requirements.  A few  generally  useful  tech- 
niques have  been  presented,  however,  there  are 
undoubtedly  other  techniques  which  can  be  used  for 
specific  system  applications  the  challenge  is  to 
design  them  in  early  and  make  them  effective  in 
achieving  greater  mission  assurance. 


Table  5.  Summary 


• Aerospace  systems  have  little  internal 
visibility  and  are  difficult  to  troubleshoot. 

• Factoring  in  test  during  system  desiqn  maxi- 
mizes visibility  and  eases  troubleshooting. 

• Automating  test  gives  repeatability,  con- 
sistency, and  effective  data  presentation. 


• Simple  command  and  telemetry  systems  maxi- 
mize testability. 

• Both  hardware  and  software  are  amenable  to 
built  in  test  features. 

• Built  in  test  features  generally  lower  con- 
ventional (component  count)  reliability 
estimates. 

• Built  in  test  features  increase  assurance  by 
more  comprehensive  testing. 


THE  DESIGNER/RESPONSIBLE  EQUIPMENT  ENGINEER'S 
ROLE  IN  THE  LIFE  CYCLE  OF  SPACE  HARDWARE 

Mr.  Robert  C.  Koche 
Program  Manager 

Lockheed  Missiles  and  Space  Company 


We  are  here  to  exchange  ideas  on  what  it  takes  to 
achieve  mission  success.  More  specifically,  our 
workshop  session  addresses  "designing  for  mission 
assurance".  To  that  end,  I would  like  to  talk  to 
a concept  that  has  evolved  at  Lockheed  Missiles  and 
Space  Company  and  which,  we  think,  is  a cornerstone 
of  our  successes.  It  is  the  concept  of  the  Respon- 
sible Eguipment  Engineer. 

To  me  the  Designer  and  Responsible  Eguinment  Engi- 
neer (REE)  are  somewhat  synonymous.  In  our  company 
he  is  the  same  in  most  instances  and  is  the  focal 
point  for  the  design  of  an  equipment  item.  Even 
though  we  may  buy  equipment  from  a subcontractor, 
the  Responsible  Equipment  Engineer  is  the  one  held 
responsible  by  supervision  and  management  for  the 
proper  design,  test,  and  to  a major  extent,  the 
quality  of  the  hardware.  It  is  the  REE  who  derives 
equipment  performance  requirements  from  the  System 
Specification  and  translates  it  into  design  require- 
ments for  the  Subcontractor.  The  REE  writes  the 
equipment  specification,  reviews  and  approves  the 
related  documentation  and  follows  the  hardware  from 
the  cradle  to  the  grave.  In  house  he  may  be  the 
man  on  the  board  or  at  the  desk  who  truly  is 
designing  every  detail  of  the  equipment.  When  I 
use  the  term  “equipment",  it  can  mean  anything  from 
a solar  array  module  to  a guidance  black  box  to  a 
structural  frame. 

The  first  slide  is  an  extract  from  our  Policies  and 
Procedures  and  shows  the  charter  established  for 
"The  Responsible  Equipment  Engineer". 

If  you  will  accept  the  fact  that  in  at  least  one 
Aerospace  Company,  the  REEs  are  the  backbone  of  the 
design  organization;  I'll  tell  you  what  we  expect 
of  them.  First,  there  are  certain  attributes  we 
think  are  common  to  all  good  REEs.  These  are 
inquisitiveness,  a good  sense  of  closure,  a quick- 
ness to  bring  in  specialist  support  when  needed, 
and  be  a veritable  encyclopedia  of  knowledge  relat- 
ing to  his  equipment.  We  expect  the  REE  to  conceive 
design  solutions  which  meet  requirements  yet  at  the 
same  time  are  simple,  cost  effective,  within  the 
state  of  art  and,  hopefully,  have  small  development 
risks. 

The  REE  interfaces  with  practically  everyone.  This 
next  slide  shows  the  range  of  interfaces.  They 
include  System  Engineers,  Flight  Science  special- 
ists, Materials  and  Process  specialists,  customer. 
Aerospace,  Product  Assurance,  Manufacturing,  test 
activities,  launch  pad  personnel  and  even  mission 
support  groups.  The  interfacing  may  be  as  little 
as  an  occassional  presentation  to  the  customer  as 
to  how  his  equipment  works,  to  as  much  as  daily 
contacts  with  Reliability,  stress  analysts, 
Thermodynamiclsts,  and  Product  Assurance  test 
technicians.  Quite  often  people  think  of  him  as 
Mr.  Horizon  Sensor,  Attitude  Reference  Assembly, 
or  some  other  such  name. 


prototyping,  qualification,  first  article 
auditing  and  then  nursing  each  flight  item  through 
each  and  every  ATP  failure.  Also,  we  expect  him 
to  know  how  his  equipment  interfaces  with  other 
equipment  (or  fits  in  the  system),  how  it  is  used 
in  flight,  its  actual  flight  environment,  which 
performance  parameters  are  critical  and  which  are 
not.  All  of  this  so  that  he  can  make  the  best  pos- 
sible judgment  during  the  design  evolution  or  when 
evaluating  test  data.  The  REE  reviews  all  test 
data  and  looks  for  trends.  We  expect  him  to  con- 
stantly improve  test  procedures,  specification 
limits  and  to  identify  needed  design  improvements 
when  required.  Maybe  we  even  expect  him  to  have 
a sixth  sense  to  know  when  to  pull  his  equipment 
from  the  satellite  vehicle  even  though  it  is 
operating  within  specification.  What  we  are  trying 
to  do  is  instill  a sense  of  craftsmanship  into  each 
of  the  REEs  and  to  cound  on  his  good  judgment  to 
supplement  the  formal  documentation.  If  we  do  this, 
a good  part  of  the  battle  for  mission  success  is 
won. 

After  we  achieve  this  level  of  performance  from 
one  individual,  the  likelihood  is  that  he  will  not 
remain  an  REE  very  long.  It's  these  types  of 
engineers  that  quickly  become  supervisors,  managers 
and  more. 

To  restate  what  I've  been  trying  to  say  is, 
"Designers  or  Responsible  Equipment  Engineers,  or 
whatever  you  want  to  call  them,  of  the  right  cali- 
ber, and  with  a total  dedication,  are  what  breed 
mission  success.  We  must  give  ample  attention  to 
their  selection,  training,  and  assignment  of 
responsibilities  and  authorities." 

SSD  Policy  Directive 
(Extract) 

A.  Policy 

SSD  organizations  having  original  design 
responsibility  shall  assign  continuing  tech- 
nical responsibility  and  authority  for  thier 
products  or  equipment  to  a "responsible 
equipment  engineer"  (REE). 

B.  Responsibilities 
3.  Responsible  equipment  engineers: 

A)  Establish  and  document  engineering 
criteria  necessary  to  assure  equip- 
ment development,  design,  testing, 
qualification,  production  or  pro- 
curement, and  application  in 
response  to  stated  program 
requirements. 

B)  Serve  as  the  centralized  source  of 
technical  information  for  the 
assigned  product  or  equipment  and 
the  focal  point  for  technical  direc- 
tion or  review  of  all  actions  per- 
taining to  design,  modifications, 
testing,  performance,  discrepancies 
and  failures,  etc. 


We  expect  the  REE  to  be  heavily  involved  in  the 
birth  of  his  equipment.  We  expect  him  to  nurse  it 
through  the  growing  pains  of  breadboarding. 
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SPACECRAFT  REDUNDANCY 

Kenneth  P.  Timmons 
Director,  Systems  Engineering 
Martin  Marietta 

The  application  of  redundancy  to  reduce  the  risk  of 
spacecraft  mission  loss  is  a common  practice.  It 
should  not,  however,  be  added  indiscriminately. 

The  intent  of  this  paper  is  to  provide  a summary 
of  those  factors  or  considerations  which,  I believe, 
have  the  greatest  influence  on  spacecraft  redun- 
dancy design.  To  that  end,  we  will  discuss  the 
philosophy  which  must  be  established  at  the  very 
beginning  of  a spacecraft  program  to  set  the  stage 
for  us  to  select  an  approach  to  redundancy  imple- 
mentation. Once  we  have  defined  and  established  a 
philosophy  and  have  settled  on  an  implementation 
approach,  we  must  employ  a methodology  which  will 
ensure  our  meeting  system  mission  goals  or  require- 
ments within  the  context  of  the  established  philos- 
ophy and  implementation  approach.  I will  illus- 
trate these  processes  with  some  examples  from  two 
Martin  Marietta  designed  and  built  spacecraft, 

SCATHA  (Space  Test  Program  Flight  P78-2)  and  Viking, 
the  Mars  Lander  vehicle.  Lastly,  I will  discuss 
the  need  for  and  the  importance  of  verifying  redun- 
dancy prior  to  the  start  of  a mission  along  with 
some  typical  limitations. 

Redundancy  Philosophy 

The  redundancy  philosophy  to  be  applied  to  a given 
project  depends  on  several  factors  including:  the 
type  of  program,  for  example,  a large  scale  produc- 
tion program  such  as  the  Copperhead  Laser  guided 
projectile  or  one-of-a-kind  program  such  as  a 
special  satellite;  the  national  priority  assigned 
to  the  program,  the  program  can  range  from  one  of 
significant  scientific  importance  or  critical  to 
national  defense  to  one  of  relatively  minor  signifi- 
cance; mission  and  system  characteristics. 

The  one  or  two  of  a kind  program  usually  infers 
lower  acceptable  risk  just  because  of  the  small 
build  quantity  where  a failure  is  more  likely  to 
end  a mission.  Redundancy  is  one  of  the  basic 
techniques  employed  to  minimize  the  risk  of  mission 
loss.  For  program  having  larger  build  quantities, 
on  the  other  hand,  a failure  can  be  overcome  by 
activating  a new  item  drawn  from  the  production 
(this  is  in  itself  a form  of  redundancy). 

National  priorities  impact  programs  by  mandating 
lower  mission  risk  as  priority  increases.  Usually, 
the  higher  the  priority  the  more  time  and  money  is 
available  to  reduce  mission  risk.  This  results 
usually  in  a more  forgivable  system  design  devel- 
oped by  performing  more  rigorous  design  analysis, 
applying  greater  safety  margins  and  protecting 
against  single  failures  which  would  cause  the  loss 
of  the  mission  by  implementing  redundancy. 

Mission  and  system  characteristics  have  great 
•ir:iIue:..e  on  the  redundancy  philosophy  to  be 
employed.  For  instance  an  orbiting  satellite  in  a 
geo- stationary  orbit  is  capable  of  basically  oper- 
ating and  being  controlled  from  the  ground  in  real- 
time. System  performance  can  be  continually  moni- 
tored and  ground  commands  issued  to  alter  system 
characteristics,  all  with  little  or  no  transmission 
delay.  The  occurence  of  an  anomaly  aboard  such  a 
spacecraft  is  usually  known  by  ground  operating 
personnel  who  can  uplink  corrective  commands  in  a 


time  span  short  enough  to  prevent  total  loss  of  the 
spacecraft.  On  the  other  hand,  we  have  the  orbital 
satellite  that  is  out  of  view  of  its  ground  control 
station  for  an  extended  period  or  the  interplane- 
tary spacecraft  which  is  so  far  from  earth  that 
radio  transmission  require  substantial  time  to 
travel  the  great  distances  involved.  The  telemetry 
signal  from  the  Viking  Lander  on  the  surface  of 
Mars  takes  a little  more  than  twenty  (20)  minutes 
just  to  reach  the  ground  stations  here  on  earth. 
Corrective  commands  from  earth  to  Mars  require  an 
equal  amount  of  time  (about  40  minutes  total). 

This  is  before  any  interpretation  of  telemetry  or 
command  generation  activity  is  initiated. 

Sometimes  when  delay  time  or  the  environment  make 
command  impossible,  as  in  the  case  of  the  Viking 
Lander,  the  spacecraft  must  be  autonomous,  that  is, 
it  must  be  capable  of  independent,  fully  auto- 
matic operation.  Conversely,  when  a spacecraft  or 
satellite  is  continuously  in  view  of  a ground  com- 
mand station,  the  spacecraft  can  be  dependent, 
i.e.,  ground  commands  may  be  used  to  perform  all 
spacecraft  control  functions.  And  then  there  are, 
of  course,  combinations  of  these.  Such  was  the 
Viking  Lander  which  could  either  operate  auto- 
nomously for  critical  periods  such  as  Martian 
atmospheric  entry  or  during  landed  operation  or, 
could  be  controlled  by  ground  command  throughout 
the  landed  mission. 

The  redundancy  philosophy  then,  which  involves 
such  considerations  as  the  use  of  standby  block 
redundancy  with  automatic  failure  sensing  and 
switchover  to  the  redundant  unit,  active  block 
redundancy,  majority  voting  schemes,  alternate 
operating  modes,  operational  work  arounds  and  other 
techniques  to  correct  or  circumvent  an  anomaly  or 
failure,  must  be  based  upon  at  least  the  considera- 
tions we've  just  reviewed.  Once  a redundancy 
philosophy  has  been  established,  the  next  step  is 
to  develop  a total  system  design  which  reflects  the 
philosophy. 

There  are  at  least  two  basic  approaches  to  redun- 
dancy implementation  that  can  be  followed  once  a 
philosophy  is  established.  The  first,  and  per- 
haps most  common,  approach  is  to  set  a numerical 
mission  success  goal  or  requirement.  Estimated 
(predicted)  system  reliability  is  compared  against 
the  numerical  goal  and,  when  the  initial  system 
design  fails  to  meet  the  goal  or  requirement,  redun- 
dancy is  added  in  accordance  with  the  established 
philosophy  until  the  goal  or  requirement  is  met. 

This  approach  helps  to  force  tradeoffs  of  the  rela- 
tive value  of  a number  of  proposed  redundancy  addi- 
tions to  achieve  maximum  reduction  of  risk  for  a 
minimum  of  other  penalties  such  as  weight  and  cost. 
The  drawback  to  this  approach  is  that  the  design  is 
being  driven  by  a numerical  qoal  or  requirement 
that,  even  if  it  is  achieved,  recognizes  that  a 
failure  could  cause  the  loss  of  the  mission.  Risks 
are  reduced  but  there  is  no  Inducement  to  eliminate 
critical  single  failure  points  which,  no  matter  how 
low  a risk  they  are,  if  they  should  occur  would 
spell  the  end  of  the  mission.  This  brings  us  to 
the  second  approach  which  I call  the  "NO  SINGLE 
FAILURE"  approach.  This  approach  involves  the 
establishment  of  a "Malfunction  Protection  Policy" 
which  forces  the  elimination  of  all  system  sirgle 
point  failures  through  the  application  of  redjn- 
dancy  in  accordance  with  the  philosophy  established 
or,  requires  those  single  failure  points  to  be 


designed  for  minimum  risk  by  applying  stress 
derating  and  design  margin.  The  derating  and  mar- 
gin would  normally  be  greater  when  protecting  a 
single  failure  point  than  when  implemented  as  a 
general  design  criteria. 

Ideally,  both  of  these  approaches  would  be  imple- 
mented. The  combination  of  the  two  ensures  that 
single  point  failures  are  identified  and  protected. 
When  the  protection  involves  redundancy,  the  best 
approach  can  be  identified  using  a numerical  trade- 
off method  to  optimize  risk  reduction  as  a function 
of  weight,  cost  or  other  critical  parameters. 

Assuming  a program  implements  a malfunction  protec- 
tion policy,  the  first  step  in  identifying  critical 
single  point  failures  is  to  perform  a failure  mode 
and  effects  analysis  (FMEA)  or,  what  is  also  termed 
a failure  mode,  effects  and  criticality  analysis 
(FMECA).  This  analysis  should  be  first  performed 
at  the  system  or  major  subsystem  level  and  should 
examine  the  functional  failure  modes  of  each  black 
box  or  end  item.  O.casional ly , the  analysis  must 
go  to  the  functional  level  in  a black  box.  For 
dependent  spacecraft  implementing  standby  block 
redundancy  with  failure  detection  and  command 
switchover  occu-ring  on  the  ground,  the  functional 
levels  of  failure  mode  analysis  are  usually  ade- 
quate. However,  when  the  spacecraft  is  capable  of 
operating  on  its  own  without  ground  intervention, 
it  is  important  to  take  the  analysis  down  to  the 
piece  part  level.  In  fact,  if  a real  understanding 
of  failure  modes  is  to  be  gained,  the  analysis  must 
go  to  the  functional  areas  of  each  integrated  cir- 
cuit chip  or  hybrid  circuit.  This  is  very  impor- 
tant since  whole  ICs  don't  fail  completely  most  of 
the  time.  Frequently,  one  gate  will  stick  high  or 
low,  one  bit  will  be  lost  in  a register,  one  of 
two  or  three  line  dirvers  will  fail  on  or  off. 

This  is  especially  true  when  the  IC  has  separate 
power  and  signal  leads  for  groups  of  functions 
contained  on  the  single  chip  and  in  a single 
package. 

Following  the  failure  mode  analysis,  a series  of 
trade  studies  must  be  performed.  These  trade 
studies  are  needed  to:  optimize  the  design 
improvement;  determine  what  must  really  be  pro- 
tected i.e.,  can  a possible  hardware  failure 
more  easily  be  protected  by  a software  or  sequence 
of  events  change;  and  lastly,  as  we  have  already 
mentioned,  optimize  the  risk  reduction  as  a func- 
tion of  weight  or  cost. 

Lastly,  after  the  trade  studies  have  been  com- 
pleted there  will,  no  doubt,  be  some  single  failure 
points  that  do  not  lend  themselves  to  protection 
by  applying  redundancy.  Typical  of  these  failure 
modes  are  primary  structure,  propulsive  hardware 
such  as  lines  and  fittings  or  thrust  chambers. 
Frequently  complex  mechanical  hardware  such  as 
steerable  solar  array  systems,  boom  drive  assem- 
blies or  steerable  antenna  systems  cannot  be  pro- 
tected by  redundancy  due  to  the  high  weight  or 
power  penalty  that  must  be  paid.  All  of  these 
types  of  sinqle  point  failures  are  then  protected 
by  applying  an  above  the  normal  design  safety  mar- 
gin. The  same  approach  applies  to  electronics 
except  the  stress  derating  would  be  increased. 

Now  let  us  take  a brief  look  at  some  examples  of 
redundancy  implementation.  I've  chosen  the  SCATHA 
(Space  Test  Program  Flight  P78-2)  orbital  satellite 


and  the  Mars  Viking  Lander  programs.  Both 
programs  have  almost  identical  malfunction  protec- 
tion requirements.  The  implementation  of  redun- 
dancy, however,  is  vastly  different.  The  dif- 
ferences are  due  to  mission/system  characteristics 
and  national  priorities. 

The  SCATHA  mission  places  the  satellite  in  view  of 
a ground  command  station  most  of  the  time.  Com- 
munications times  are  also  short  which  allows 
almost  continuous  ground  monitoring  of  spacecraft 
performance  along  with  the  ability  to  uplink  a 
ground  command  to  correct  an  anomaly  whenever 
required.  The  ability  to  control  the  satellite  in 
what  amounts  to  real  time  permitted  the  applica- 
tion of  a redundancy  philosophy  centered  around  the 
concept  of  standby  block  redundancy.  Exceptions 
are  the  cormand  system,  which  must  be  continuously 
accessible,  and  the  experiments.  The  redundancy 
implementation  approach  included  a functional 
FMEA  down  to  the  black  box  level.  As  a result  of 
this  analysis,  20  black  box  functions  were  pro- 
tected by  redundancy  while  four  are  protected  by 
design  margin.  All  functional  black  box  redundancy 
is  verified  prior  to  mating  of  the  satellite  with 
the  payload  fairing.  Also,  provisions  are  included 
to  automatically  save  the  satellite  by  removing 
all  non-command  critical  loads  from  the  power  sub- 
system in  the  event  of  an  excessive  power  drain. 

The  system  can  be  reactivated  by  gound  command  at 
which  time  the  anomaly  causing  the  problem  can  be 
circumvented  by  utilizing  the  available  block 
redundant  hardware.  Again,  this  is  possible  due 
to  basically  real  time  telemetry  and  command 
capability. 

Before  proceeding  to  the  Viking  Lander,  I want  to 
take  just  a minute  or  so  to  illustrate  the  actively 
redundant  and  cross-strapped  command  system 
employed  in  SCATHA.  Both  receivers  and  command 
decoders  are  powered  on  continuously.  Both 
receivers  drive  both  command  decoders.  Isolation 
is  used  at  the  decoder  inputs,  for  example  to  pre- 
vent a shorting  of  one  input  of  a decoder  from 
reflecting  to  and  affecting  the  other  decoder.  The 
same  is  true  in  the  case  of  any  open  circuit.  No 
failure  of  either  command  receiver  or  either  com- 
mand decoder  will  cause  the  loss  of  commands  or  the 
generation  of  spurious  commands.  The  cross- 
strapped  receiver/decoder  elements  drive  block 
redundant  command  distribution  units  to  provide 
fully  protected  command  capability. 

Now  for  a discussion  of  the  Viking  System.  Here 
we  have  a system  that,  because  of  the  extremely 
long  delay  in  communications  coupled  with  the  need 
to  fly  a difficult  Martian  atmospheric  entry  with- 
out ground  control,  was  fully  autonomous.  A gen- 
eral purpose  digital  computer  controlled  the  Viking 
l.ander  beginning  with  separation  from  its  orbital 
mate,  through  plaentary  entry,  achieved  a soft 
landing  and  then  performed  the  landed  mission. 
Because  of  the  fully  automatic  nature  of  opera- 
tion, it  was  necessary  to  understand  fully  the 
effect  of  functional  failures  down  to  and  including 
functional  elements  within  microcircuits.  There- 
fore the  FMEA  was  performed  at  the  sub  piece  part 
level.  The  redundancy  philosophy  included  the  use 
of  active,  standby  and  cooperative  redundancy  as 
well  as  operational  workarounds.  On  Viking,  23 
black  box  functions  were  protected  by  redundancy 
while  28  were  protected  by  design  margin.  Items 
protected  by  margin  only  are  propulsive  elements 


such  as  thrust  chambers,  the  aerodecelerator 
(parachute),  the  aeroshell  and  other  similar  items. 
In  both  SCATHA  and  Viking  the  experiments  themselves 
were  not  protected  by  redundancy.  Viking  as  SCATHA, 
verified  its  redundancy  at  the  latest  possible 
point  prior  to  launch  and  included  an  onboard  capa- 
bility to  save  the  system  in  the  event  of  an  exces- 
sive power  drain. 

This  leaves  us  with  one  item  that,  although  we've 
already  touched  on  it  briefly,  I would  like  to 
emphasize.  That  is  the  need  to  verify  whatever 
redundancy  is  employed  at  the  latest  point  possible 
prior  to  starting  a mission.  This  is  mandatory  if 
the  highest  confidence  is  to  be  achieved  in  the 
spacecraft.  Of  course,  there  are  practical  limita- 
tions. For  example,  redundant  black  boxes  can  be 
individua-ly  verified  only  if  independent  switching 
is  employed  to  control  these  boxes.  On  the  other 
hand,  redundancy  implemented  at  the  part  level, 
such  as  parallel  fuses,  relay  contacts,  can  be 
verified  only  prior  to  installation  on  the  printed 
circuit  board.  Once  installed,  individual  access 
to  these  parts  is  usually  lost.  The  key,  however, 
is  to  verify  all  redundancy  at  the  latest  point 
possible  - don't  take  it  for  granted  that  its  there 
ready  to  go  to  work  for  you. 

This  completes  my  discussion  on  spacecraft  redun- 
dancy. Thank  you  for  your  attention  and  interest. 

Redundancy  Philosophy 

Depends  on: 

• Type  of  program  - e.g.,  large  scale 
production  vs  one  of  a kind 

• National  priority  - priority  level 
influences  dollars  and  schedules 

• Mission  and  system  characteristics 
Redundancy  Implementation  Approach 

• Redundancy  employed  to  meet  numerical 
goal  or  requirement 

• Forces  trade-offs  to  optimize  risk 
reduction 

• Can  permit  mission  critical  failure 
modes 

t Redundancy  added  in  accordance  with  a 
"Malfunction  Protection  Policy" 

• Helps  to  guarantee  that  all  mission 
critical  failure  modes  are  protected 

• Identifies  those  failures  that  can 
only  be  protected  by  means  other 
than  redundancy 

• Combinations  of  the  above  (preferred) 
Redundancy  Implementation  Methodology 

• Perform  a failure  mode  and  effects 
analysis* 


• Functional  to  black  box  level 

• Functional  within  a black  box 

• Piece  part  level 

• Functional  level  within  a piece  part 

• Use  trade  studies  to: 

• Optimize  design  approach 

• Determine  what  must  really  be 
protected 

• Optimize  risk  reduction  vs  weight 
and  cost 

• Flat  remaining  single  point  failures 

• Justify  remaining  single  point 
failures 

• Show  what  technique  is  used  to 
reduce  risk  of  remaining  single 
point  failures 

Redundancy  Implementation  Examples 
SCATHA/Vikinq  Lander 

• Both  programs  have  similar  requirements: 

"No  single  point  failure  shall  cause  the 
loss  of  ..."  (SCATHA) 

"No  single  failure  shall  cause  the  loss 
of  ..."  (Viking) 

• Redundancy  Implementation  was  different 
between  programs  due  to: 

• Mission  and  system  characteristics 

• National  priorities 

SCATHA 

Dependent  system  performance  is  monitored  on  the 
ground  with  redundancy  switching  by  ground  com- 
mand (except  for  critical  command  functions). 

• FMEA  performed  to  functional  black  box 
level 

• Block  redundancy  implemented  at  black 
box  level* 

• 20  black  box  functions  are  redundant 

• A black  box  functions  are  non- 

non- redundant 

• All  redundant  black  boxes  are  verified 

verified  prior  to  mating  of  space- 
craft with  payload  fairing 

• Onboard  capability  to  save  the  system  in 
the  event  of  excessive  power  system  load- 
ing. Ground  commands  required  to  correct 
anomoly. 


‘Depth  of  FMEA  depends  on  type  of  spacecraft  

(i.e.,  autonomous/dependent)  *A1 1 experiments  are  non-redundant 
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• Little  delay  between  the  transmission  and 
receipt  of  telemetry  data  or  commands. 


Command  Receiver  Cross-Strapping 

No  failure  of  either  receiver  or  decoder  will  cause 
the  loss  of  commands  or  generation  of  spurious 
commands. 


To  Block 

Redundant  Command 
Distribution  Units 


Viking 

Autonomous  system-  fully  automatic  control  by  a 
general  purpose  digital  computer. 

• Very  long  delay  between  transmission  and 
receipt  of  data  (20  minutes  one  way) 

• FMEA  performed  at  black  box,  piece  part 
and  sub  piece  part  level 

• Active,  standby  and  cooperative  redun- 
dancy* employed  as  well  as  operational 
workarounds 

• 23  black  box  functions  were 
redundant 

• 28  black  box  functions  were 
non-red-ndant 

• All  redundant  black  boxes  were 
verified  prior  to  encapsulating  in 
bioshield  and  payload  fairing  mating 

• Onboard  capability  to  save  the  system  in 
the  event  of  excessive  power  subsystem 
loading.  Ground  comnands  required  to 
correct  anomoly. 


*A11  experiments  are  non-redundant 
Redundancy  Verification 

Redundancy  must  be  verified  as  close  to  start  of 
mission  as  possible  if  highest  confidence  is  to  be 
achieved. 

• Block  redundancy  should  always  be  tested 
and  verified  at  black  box  function  level 

• Piece  part  redundancy  (e.g.,  parallel 
fuses,  relay  contacts,  etc.)  usually  can- 
not be  verified 
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Design  Workshop 
Process  Description 


Conclusion:  Establish  SAMSO/ Indus  try  Panel  to 
Examine  Relative  Merits 


• 4 Subgroups  of  8-12  Attendees  Formed 

• Design  Issues  Identified 

t Central  Themes  Compiled 

• Recommendations  Formulated 

F i nd i ng  s/ Recommenda  t i on  s 

(1)  Early,  Stable  Definition  of  Requirements 

• Review  of  Draft  RFP 

(2)  Design  Review 

• Contractor  Independent  Internal 

• Customer  DR  Not  Be  For  Show 

• Aerospace  Responsibility  Needs  Re- 
Examined 

• Improved  Software  FMEA  Methods  Needed 


(8)  Reliability  Analysis  Most  Beneficial  Technique 

• Current  Programs  Generally  Satisfactory 

• Extend  to  Mechanical,  Electromechanical, 
Hydraulics  and  Software 

• Application  to  Older  Hardware 

• Extend  FMEA  to  Manufacturing,  Test  and 
Mission  Analysis  Phases 

Other  Items 

• Failure  Data  Base  Dissemination 

a MIL  STDS/Specs  - Discussion  Forum 


(3)  Constraints  on  Resources,  Schedule,  Cost, 
Personnel  and  Performance 

• Weighting  Factors  Needed 

e Cost  Pressure  Significant  Detriment 

(4)  Issue:  Continuity  of  People 

• Government  Project  Officers 

• Design  Engineering  Needed  Through 
Hardware  Phase 

• Sustaining  Engineering  Costs 

• Motivation  for  Equipment  Engineers 

(5)  Design  Specialists 

• Effective  Technique  for  Improving  Design 
for  Mission  Assurance 

• Industry  Policies  Needed 

• Cost  Pressures  Discourage  Implementation. 

(6)  Design  Deficiencies  Major  Cause  of  Cata- 
strophic Failure 

• Emphasis  Required  in  Design  Process 

Interdisciplinary  Skills 

• Continuing  Independent  Design  Examination 

• Electromechanical,  Mechanical  Assemblies 
Primary  Source  of  Design  Problems 

Critical  Item  Control  Needed 

(7)  Launch  of  Prototype  Spacecraft/Hardware 

• Cheap  Ride  (Shuttle) 

• Recovery 

• Failure  Detection/Correction  Prior  to 
Operational  Deployment 

• Check  Out  Prior  to  Transfer  Orbit 

• Test  Article  Characteristics 
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DESIGN  FOR  MISSION  ASSURANCE 

QUESTIONNAIRE 

(32  Respondents) 

Yes 

No 

No  Opinion 

1. 

Are  environmental  criteria/requirements  over-specified? 

n/ 

17/ 

4/13% 

34“ 

53% 

2. 

Are  design/development  schedules  adequate? 

7/ 

22/ 

3/8% 

23% 

69% 

3. 

Is  the  current  system  for  defining  booster  weight-lifting 

12/ 

2/ 

18/56% 

capability  adequate? 

38% 

6% 

4. 

Should  the  hardware  designer  be  involved  in  manufacture 

30/ 

2/ 

0 

and  test? 

94% 

6% 

5. 

Are  adequate  provisions  for  testability  incorporated  in 

12/ 

17/ 

3/8% 

design? 

38% 

54% 

6. 

Should  formal  product  design  reviews  constrain  design 

25/ 

3/ 

4/13% 

release? 

79% 

8% 

7. 

Is  AGE  compatibil ity  with  flight  hardware  adequately 

8/ 

19/ 

5/16% 

reviewed? 

25% 

59% 

8. 

Does  inability  to  test  flight  hardware  for  difficult 

15/ 

13/ 

4/13% 

environments  engender  excessive  risk? 

47% 

40% 

9. 

Is  risk  identification  and  management  properly  implemented? 

10/ 

18/ 

4/13% 

31% 

56% 

10. 

Is  adequate  emphasis  placed  on  Electrical/Mechanical  AGE 

8/ 

18/ 

6/19% 

design? 

25% 

56% 

11. 

Does  use  of  redundancy  decrease  emphasis  on  good  design? 

9/ 

20/ 

3/8% 

29% 

63% 

12. 

Are  drawing  releases  demanded  too  early  in  a program? 

15/ 

12/ 

5/16% 

47% 

37% 

13. 

Are  design  changes  effectively  screened  and  controlled? 

18/ 

13/ 

1/3% 

56% 

41% 

14. 

Are  design  performance  budgets  worth  their  upkeep? 

21/ 

2/ 

9/28% 

66% 

6% 

15. 

Is  adequate  attention  paid  to  interface  definition  and 

20/ 

8/ 

4/12% 

control ? 

63% 

25% 

16. 

Should  the  prime  contractor  exercise  more  control  of 

23/ 

7/ 

2/6% 

sub-contractors'  design/development  activities? 

71% 

23% 

17. 

Are  hardware  designers  sufficiently  versed  in  orbital 

7/ 

19/ 

6/18% 

operations? 

23% 

59% 

18. 

Should  more  emphasis  be  placed  on  test  in  lieu  of  analyses 

19/ 

9/ 

4/13% 

for  performance  verification? 

59% 

28% 

19. 

Are  performance  requirements  by  the  Government  over-specified? 

12/ 

13/ 

7/23% 

37% 

40% 

20. 

Are  structural  design  margin  requirements  (margins  of  safety) 

0 

20/ 

12/37% 

excessive? 

63% 

21. 

Should  FMEA  incorporate  sneak  path  analysis? 

25/ 

1/ 

6/18% 

i 

79% 

3% 

22. 

Do  you  use  a preferred  parts  list? 

28/ 

2/ 

2/6% 

I 

88% 

6% 

J " 

QUESTIONNAIRE 


Yes 

No 

No  Opinion 

23. 

Is  the  parts  selection  process  overly  restrictive  of 

2/ 

25/ 

5/15% 

your  design? 

6% 

79% 

**24. 

What  percentage  of  the  parts  selected  by  your  normal 

10/ 

2/ 

20/63% 

process  prove  to  be  “poor"  parts? 

31% 

6% 

25. 

Is  your  current  parts  selection  process  cost  effective? 

15/ 

7/ 

10/31% 

46% 

23% 

26. 

In  your  personal  experience,  are  part  failures  a 

19/ 

10/ 

3/9% 

principal  cause  of  mission  failures? 

60% 

31% 

27. 

In  your  own  personal  experience,  are  failures  due  to 

14/ 

13/ 

5/16% 

part  misapplication  an  important  part  of  mission  failure? 

44% 

40% 

28. 

Do  you  use  reliability  prediction  as  a portion  of  your 

25/ 

5/ 

2/6% 

design  process? 

78% 

16% 

29. 

Does  your  reliability  prediction  work  encompass  stress 

24/ 

4/ 

4/12-1/2% 

derating? 

75% 

12-1/2 

30. 

Do  you  use  a failure  modes  effects  and  criticality 

28/ 

2/ 

ro 

cr. 

analysis  in  your  design  process? 

88 

6 

31. 

Is  the  failure  modes  analysis  effective  in  identifying 

22/ 

4/ 

6/18% 

redundancy  requirements,  single  point  failure  modes,  or 
other  design  weaknesses? 

69 

13% 

32. 

Have  you  experienced  mission  failure  due  to  single  point 

6/ 

22/ 

4/13% 

failure  modes  that  were  identified  during  the  design 
process? 

18% 

69% 

33. 

Have  you  experienced  mission  failure  due  to  previously 

10/ 

17/ 

5/16° 

unidentified  single  point  failure  modes? 

31% 

53% 

34. 

Are  you  satisfied  with  your  current  design  review 

12/ 

16/ 

4/13% 

practices? 

37% 

50% 

35. 

Is  the  customer's  participation  in  your  design  review 

25/ 

3/ 

4/13% 

beneficial ? 

79% 

8% 

36. 

Do  you  use  a formal  failure  analysis  and  correction 

27/ 

3/ 

2/6% 

action  program? 

84% 

8% 

37. 

Is  your  failure  analysis  and  corrective  action  program 

26/ 

3/ 

3/8% 

effective  in  preventing  failures  of  a similar  kind? 

84% 

8% 

38. 

Are  you  satisfied  with  your  current  design  change 

18/ 

13/ 

1/3% 

evaluation  techniques? 

56% 

41% 

39. 

Have  you  had  mission  failures  due  to  poor  change  evaluation? 

5/ 

24/ 

3/9% 

16% 

75% 

40. 

Are  you  satisfied  with  the  translation  of  design  data  to 

9/ 

19/ 

4/12% 

manufacturing  instructions? 

28% 

60% 

41. 

Are  you  satisfied  with  your  current  design  techniques  for 

12/ 

12/ 

8/25% 

moving  mechanical  assemblies? 

37-1/2%, 

37-1/2% 

42. 

Have  you  experienced  mission  failure  of  a moving 

17/ 

14/ 

1/3% 

mechanical  assembly? 

53% 

44% 

43. 

Are  you  satisfied  with  your  current  thermal  design 

17/ 

11/ 

4/13% 

techniques? 

53% 

34% 

44. 

Have  you  experienced  a mission  failure  due  to  thermal  design? 

6/ 

22/ 

4/13% 

69% 

18% 

I 
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QUESTIONNAIRE 


Yes 

No 

No  Opinion 

45. 

Are  you  satisfied  with  your  current  material  and  process 

21/ 

10/ 

1/3% 

selection  and  application  techniques? 

66% 

31% 

46. 

Have  you  experienced  problems  due  to  differences  between 

25/ 

6/ 

1/3% 

government/industry  interpretation  of  design  requirements? 

79% 

18% 

47. 

Are  you  satisfied  with  the  FCRC  (Aerospace,  Mitre,  et  al) 

8/ 

15/ 

9/28% 

role  in  your  design? 

25% 

47% 

48. 

Does  your  company  or  organization  utilize  a special 

12/ 

14/ 

6/18% 

motivation,  recognition  or  incentive  program  to  enhance 

38% 

44? 

mission  assurance? 


Yes 

No 

No  Opinion 

— 

**24.  5* 

Very  few 

1% 

Less  than  1% 

10% 

20% 

5% 

10% 

<1% 

<1% 

10% 

5% 

5 - 
30% 

<2% 

10% 

V.  Actual  percentages  given 
f on  some  papers 
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WORKSHOP  B 

THE  ROLE  OF  CONTRACTUAL  INCENTIVES  IN  MANAGEMENT  MOTIVATION 

FOR  MISSION  ASSURANCE 

Co-Chairmen 

Lt.  Col.  Kenneth  JuVette 

Director  of  Procurement 

Space  Navigation  Systems 

SAMSO/PMN 

Mr.  Richard  M.  Randall 

Director,  Business  Management 

McDonnell  Douglas 

Mr.  Robert  H.  Crotser 

Deputy  Program  Manager 

Space  Systems  Div. 

Coordinator 

Maj.  Robert  Dalrymple 

Asst  Director  of  Procurement, 

Space,  Comm.,  SAMSO 

AGENDA 

Wednesday,  April  26 

0830-0900 

Introduction 

Lt.  Col.  Kenneth  JuVette,  SAMSO 

0900-0920 

Review  of  Contract  Types 

Mr.  Robert  H.  Crotser,  Lockheed,  LMSC 

0920-0940 

The  Value  of  Incentives 

Mr.  Patrick  R.  McGinnis,  McDonnell  Douglas 

0940-1000 

Break 

1000-1130 

Are  Incentives  Useful  to 

Achieve  Mission  Success 

Mr.  Harold  E.  Sharp,  SAMSO 

Mr.  Grant  Hansen,  SDC 

Mr.  Robert  J.  1 ngersoll,  Boeing  Aircraft  Co. 

1130-1200 

Open  Forum 

1200-1245 

Lunch 

1245-1415 

Organizational  Effect  of 

Administering  Incentives 

Mr.  Hubbert  L.  O'Brien,  TRW  Inc. 

Col.  John  J.  Caulfield,  SAMSO 

Mr.  George  S.  Pappas,  Ball  Brothers 

1415-1430 

Break 

1430-1530 

Incentive  Innovations 

Lt  Col.  Michael  M.  McMillan,  HQ  AFSC 

Mr.  P.  Nino  Noal,  Hughes  Aircraft  Co. 

1530-1630 

Open  Discussion 

INTRODUCTION 


1415-1445  Forum 


Lt  Col  Kenneth  JuVette 
Director  of  Procurement 
Space  Navigation  Systems 
SAMSO/PMN 


Lt  Col  JuVette  welcomed  the  approximately  fifty 
attendees  and  presented  a series  of  viewgraphs  giv- 
ing the  background  and  highlights  of  the  SAMSO/ 
Industry  Executive  Session  on  Mission  Assurance  as 
it  addressed  Contractual  Incentives.  His  presenta- 
tion also  included  a series  of  unattributed  quotes 
from  letters  to  General  Morgan  written  by  industry 
representatives.  Specific  mention  was  made  of  some 
Air  Force  Lessons  Learned  in  the  area  of  Award  Fee 
and  of  the  Air  Force  Systems  Command  Initiative 
No.  13  to:  Develop  Innovative  Procedures  to  Moti- 
vate Contractors  to  Improve  Weapon  Systems  Per- 
formance In  the  Field. 


1445-1500  Break 

1500-1600  Incentive  Lt  Col  Mike  McMillan 

Innovations  AFSC,  Procurement  and 

Manufacturing 
Mr.  Nino  Noal 
Manager  of  Contracts 
Defense  Systems 
Division,  Hughes 

1600-1630  Open  Forum 


Question?? 

Is  the  present  role  of  contractual 
incentives  a positive  or  negative  force 
in  the  attainment  of  mission  success? 


The  Workshop  speakers  and  topics  were  introduced  to 
the  audience.  We  were  all  challenged  by  Lt  Col 
JuVette  in  his  closing  remarks  to  identify  counter- 
productive policies,  actions,  and  practices,  and  to 
make  recommendations  for  improving  the  Mission 
Success  of  our  space  and  missile  programs. 


Workshop  Schedule 


Background  on  Workshop 

t Meetings  were  held  during  1977  with  SAMSO 
commander  and  key  industry  executives. 

• Leaders  collectively  identified  a wide 
range  of  key  elements  critical  to  mission 
success. 


0830-0900  Introduction  Lt  Col  Ken  JuVette 

0900-0920  Review  of  Mr.  Bob  Crotser 

Contract  Types  Deputy  Program  Manager 
Lockheed  Missiles  & 
Space  Co. 

0920-0940  The  Value  of  Mr.  Dick  Randall 

Incentives  Director  of  Business 

Management,  McDonnell 
Douglas  Astronautics  Co. 

0940-1000  Break 

1000-1130  Are  Incentives  Mr.  Hal  Sharp 

Useful?  Chief,  Pricing  Office 

Mr.  Grant  Hansen 
President  SDS  Systems 
Group 


a Three  industry  associations  accepted 

along  with  SAMSO  to  explore  issues  in 
greater  detail. 

• Contractual  incentives  identified  as  a 

key  area  to  control  risk  through  improved 
management. 

Additionally  — Air  Force  systems  command  has 
established  a command  initiative  No.  13  to: 

Develop  innovative  procedures  to  motivate 
contractors  to  improve  weapon  systems 
performance  in  the  field. 

Also  - Prime  and  subcontractors  are  continually 
addressing  the  subject  of  incentives, 
motivation,  and  mission  assurance. 


Bob  Ingersoll,  Boeing 
Contracts  Manager 
Missiles  and  Space 
Division 


Mr.  Hub  O'Bryan 
TRW,  Manager  of  Con- 
tracts, Space  Sy, terns 
Division 

Col  Jack  Caulfield 
Director  of  Procurement 
for  Space  Communication 
Systems 

Mr.  George  Pappas 
Ball  Brothers,  Director 
of  Contracts 


Highlights  of  SAMSO/ Indus  try  Executive  Session  On 
Mission  Assurance  - Incentives 

• Performance  oriented  incentives  signifi- 
cantly greater  than  cost  or  schedule  are 
considered  a strong  tool  for  focusing 
management  attention  on  those  areas  that 
enhance  the  probability  of  mission 
success. 

• Properly  drawn,  incentives  could  be  the 
guide  for  every  management  decision  and 
trade-off  in  deciding  how  and  where  to 
apply  resources. 

» Achieving  the  appropriate  balance  of 

contract  incentives  is  considered  very 
Important  in  motivating  contractor 
management  in  the  right  direction. 
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Several  examples  of  how  incentives  can 
actually  contribute  to  failures, 
anomalies  and  performance  degradation 
were  given. 

Timing  of  incentive  payments  is  also 
important  to  maximize  the  motivation 
desi red. 


All  the  motivators  likely  to  affect  the 
contractor  must  be  considered  in  making 
the  decision  to  use  award  fee.  As  an 
example,  if  the  successful  accomplishment 
of  a particular  contract  will  result  in  a 
large  production  follow-on,  the  contractor 
may  be  so  motivated  by  such  a situation 
that  award  fee  would  not  be  necessary. 


Contract  incentives  can  be  used  by 
management  to  motivate  personnel  to  avoid 
practices  that  might  inadvertently  degrade 
the  product.  Incentives  should  be  passed 
down  to  the  first  line  supervisors  or 
lower  in  "real  time."' 

Both  parties  should  have  a clear  under- 
standing of  the  incentive  structure  before 
starting  work.  Rules  should  be  published 
in  advance.  Also,  there  should  be  no 
conflict  between  incentives,  i.e.,  cost 
schedule  vs.  performance. 

The  subject  of  personnel  motivation  could 
be  closely  associated  with  contract 
incentives. 

Using  the  incentive  resources  to  reward 
personnel  performance,  and  getting  it 
passed  on  to  the  man  or  woman  on  the 
bench,  is  a very  important  aspect  of 
mission  assurance. 


• In  a recent  USAF  acquisition  study  a 
questionnaire  of  industry  on  various 
procurement  practices  revealed  their 
unanimous  favor  of  award  fees.  There 
are  two  main  reasons  for  this: 

• The  contractor  liked  the  "Report  Card" 
and  found  it  was  to  be  extremely  useful 
to  his  program  manager. 

• It  was  a means  of  forcing  strong  com- 
munication links  between  contractor 
and  Government. 

Industry  Viewpoints  Expressed  To  SAMSO  Commander 

• "...I  have  puzzled  as  to  whether  or  not 
there  might  be  more  effective  incentives 
for  contractors. " 

• "...I  know  some  of  the  problems  in  trying 
to  design  a financial  incentive  package 
which  pays  off  to  the  people  whom  you 
really  wish  to  motivate." 


• Some  attendees  felt  that  multi-year  on- 
orbit  performance  incentives  that  are 
paid  downstream  are  not  effective.  The 
performance  of  contract  personnel  is 
graded  in  the  present,  not  in  the  future. 

• Early  payment  of  incentives  when  initial 
performance  is  achieved  is  effective.  We 
might  consider  the  use  of  incentives  in 
the  front  end  of  the  program  and  assess 
penalties  if  full  performance  is  not 
attained. 

• Award  fees,  properly  structured  and 
operated,  are  a good  management  tool. 

Both  parties  should  have  commonality  of 
purpose  and  mutual  understanding  of  the 
award  fee  plan. 

AWARD  FEE 

Some  USAF  Lessons  Learned 


• "The  most  powerful  incentive  is  that  for 
future  contracts,  rather  than  profit." 

"Increased  use  should  be  made  of  flight 
performance  incentives  with  greater 
rewards  and  penalties  provided. " 

We  would  also  recommend  re-establishing 
substantial  flight  performance  incentives 
on  current  and  future  programs.  I am 
convinced  that  this  type  of  incentive 
tends  to  focus  and  discipline  the  manage- 
ment attention  which  is  vital  to  mission 
success. 

• "Contractual  orbital  performance  incen- 
tives which  are  greater  than  cos*-  and 
schedule  incentives  make  it  very  clear  to 
the  contractor  that  performance  is  the 
most  important  factor  to  the  space 
customer. " 


• R&D  contracts:  Award  fee  should  be  used 
with  caution  in  such  cases  since  the 
incentf vizing  of  dmini strati ve  features 

ost,  timeliness  of  reports,  etc.)  may 
♦use  the  contractor  to  emphasize  these 
■ •••a-,  at  the  expense  of  the  R&D  effort. 

• ■*',  !■,  aitii  | high  proportion  of  labor 
»» • mmrr  s i e i fully  utilizing  award 

«»»..•  ► *»  fee  was  passed  on  to 

• t , the  on t rat  tor. 

• • m-  >«  ■ .1*  «<rf  fee  provisions 
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• "With  strong  performance  incentives,  a 
contractor  is  more  likely  to  balance 
schedule  and  cost  performance  to  take 
actions  which  the  contractor's  program 
manager  believes  will  significantly 
increase  the  probability  of  earning 
improved  performance  incentives." 

• "Fixed  price  contracts  to  the  prime,  and 
from  the  prime  to  the  subcontractors  are 
not  conducive  to  high  quality  products  in 
many  cases.  More  creative  incentive 
procurement  effort  is  desirable." 

• 'executive  managenent  participation  could 
b«  encouraged  by  including  Incentives  in 

■ract  for  performance,  using  the 


performance  records  as  part  of  the  incen- 
tive criteria  for  future  contracts,  and 
publicizing  when  a good  product  is 
produced. 

• "If  performance  is  desired,  place  the 
significant  incentive  on  mission 
performance. " 

• "Judge  both  the  Air  Force  team  and  the 
Contractor  teams  on  whether  the  incentive 
is  paid  in  full  for  full  achievement, 
partially  paid  for  mediocrity,  or  no 
incentive  for  failure." 

• "The  performance  incentive  is  a strong 
tool  for  focusing  management  attention;  it 
likewise  could  be  an  anprooriate  gauge  for 
grading  the  Air  Force  team  and  focusinq 
their  attention." 

• "If  the  performance  incentive  provision  is 
appropriately  drawn,  then  it  forms  the 
guide  for  every  management  and  technical 
decision,  i.e.,  how  does  each  particular 
decision  or  course  of  action  affect  the 
attaining  of  the  performance  fee?" 

Chal lenge 

Mission  success  is  the  overriding  goal  of 

SAMSO/ Aerospace/Contractor  teams . 

• Any  actions,  policies  or  practices  that 
tend  to  defer  from  obtaining  this  objec- 
tive are  suspect. 

• Effort  is  continuing  at  identifying  and 
changing  management  practices  that  are 
considered  counter  productive  to  achieve- 
ment of  mission  success. 


REVIEW  OF  CONTRACT  TYPES 

Mr.  Robert  H.  Crotser 
Deputy  Program  Manager 
Space  Systems  Division 
Lockheed  Missile  and  Space  Corp. 

Mr.  Crotser  presented  a series  of  cleverly  cartooned 
viewgraphs  in  a capsule  course  on  contract  types  and 
considerations.  He  demonstrated  how  the  choice  of 
contract  types  is  a function  of  risk  and  provided 
some  basic  principles  and  factors  to  be  considered 
when  selecting  a contract  type.  Mr.  Crotser  pro- 


vided a description  and  application  for  all  the 
contract  types  and  gave  some  personal  observations 
in  the  incentive  area.  Mr.  Crotser  mentioned  the 
use  of  the  "Martin  Incentive"  (a  form  of  incentive 
payment  where  the  total  performance  fee  is  paid  to 
the  contractor  upon  delivery  and  the  contractor 
must  pay  back  to  the  Government  any  amount  lost  for 
less  than  nominal  performance  during  operation)  and 
stated  that  he  expects  to  see  more  CPIF/AF  (combin- 
ation incentive  fee  and  award  fee)  contracts  in  the 
future.  Mr.  Crotser  favors  an  award  fee  contract 
in  which  the  award  fee  criteria  can  be  changed  on 
a period-to-period  basis. 


TYPES  OF  CONTRACTS 


[fixed  pric£ 

• FIRM  FIXED  PRICE 

• FIXED  PRICE  WITH  ESCALATION  PROVISIONS 

• FIXED  PRICE  INCENTIVE 

• PROSPECTIVE  PRICE  REDETERMINATION  AT  SPECIFIED  TIMES  DURING  PERFORMANCE 

• CEILING  PRICE  WITH  RETROACTIVE  PRICE  DETERMINATION  AFTER  COMPLETION 

[cost  • v>:  : 

• COST  NO  FEE 

• COST  SHARING 

• COST  PLUS  INCENTIVE  FEE 

• COST  PLUS  FIXED  FEE 

• COST  PLUS  AWARD  FEE 

• COST  PLUS  A PERCENTAGE  OF  COST 
- NOT  LEGALLY  ENFORCEABLE 

COWESI 

• TIME  AND  MATERIAL 

• LABOR  HOUR 


METHOD  OF  CONTRACTING 
-A  FUNCTION  OF  RISK  - 


fcuygi? 

SELLER 


SELLER 

BUYER 


SELECTION  OF  CONTRACT  TYPE 


\ . 


r^.^dPjisj 


• PROFIT,  GENERALLY,  IS  THE  BASIC  MOTIVE  OF  THE  BUSINESS  ENTERPRISE. 

• THE  CONTRACTING  PARTIES  SHOULD  SEEK  TO  NEGOTIATE  AND  USE  THE 
CONTRACT  TYPE  BEST  CALCULATED  TO  STIMULATE  OUTSTANDING 
PERFORMANCE 


- OBJECTIVE:  CONTRACT  TO  ENSURE  THAT  EFFECTIVE  AND  ECONOMICAL 
PERFORMANCE  IS  MET  BY  HIGH  PROFITS,  MEDIOCRE 
PERFORMANCE  BY  MEDIOCRE  PROFITS,  AND  POOR  PER- 
FORMANCE BY  LOW  PROFITS  OR  LOSSES. 


• WHERE  EFFECTIVE  PRICE  COMPETITION  IS  FEASIBLE,  THERE  IS  REASONABLE 
ASSURANCE  THAT  THE  CONTRACT  PRICE  REPRESENTS  A REALISTIC  PRICING 
STANDARD,  INCLUDING  A PROFIT  FACTOR  WHICH  REFLECTS  AN  APPROPRIATE 
RETURN  TO  THE  CONTRACTOR  FOR  THE  FINANCIAL  RISK  ASSUMED  IN 
UNDERTAKING  PERFORMANCE  AT  THE  COMPETITIVE  PRICE. 


• ABSENT  EFFECTIVE  PRICE  COMPETITION,  THE  CONTRACT  TYPE  SELECTED 
SHOULD  PROVIDE  FOR  A PROFIT  FACTOR  THAT  WILL  TIE  PROFITS  TO  THE 
CONTRACTOR'S  EFFICIENCY  IN  CONTROLLING  COSTS  AND  MEETING  DESIRED 
STANDARDS  OF  PERFORMANCE,  RELIABILITY,  QUALITY,  AND  DELIVERY. 


SELECTION  OF  CONTRACT  TYPE 


^FACTORS  TO  BE  CONSIDERED:- 

• ABILITY  TO  ESTIMATE  COSTS  FOR  THE  WORK  TO  BE  ACCOMPLISHED 

- UNCERTAINTIES  IN  TECHNOLOGY 

- TYPE  AND  COMPLEXITY  OF  ITEM/PERFORMANCE  CONTRACTED  FOR 

- STABILITY  OF  DESIGN 

- PE  Rl  OD  OF  PE  RFORMA  NC  E 

- CONTRACTOR  EXPERIENCE  WITH  SIMILAR  PROCUREMENTS 

- ADEQUACY  OF  CONTRACTOR'S  ESTIMATING  SYSTEM 

| 

• URGENCY  OF  REQUIREMENT 

• ADEQUACY  OF  CONTRACTOR'S  ACCOUNTING  SYSTEM 

• TYPE  AND  SIMILARITY  WITH  CONTRACTOR'S  CONCURRENT  CONTRACTING  EFFORTS 
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EFFECTIVE  COST  ESTIMATING 
-A  MUST  TO  INCENTIVE  CONTRACTING  - 


FIRM  FIXED  PRICE  CONTRACT 


[~DESC".IPTION^ 


PRICE  IS  NOT  SUBJECT  TO  ANY  ADJUSTMENT  BY  REASON  OF  THE  COST  EXPERI- 
ENCE OF  THE  CONTRACTOR  IN  THE  PERFORMANCE  OF  THE  CONTRACT 

PUCES  MAXIMUM  RISK  ON  CONTRACTOR  - ASSUMES  FULL  RESPONSIBILITY 
IN  THE  FORM  OF  PROFITS  OR  LOSSES,  FOR  ALL  COSTS  UNDER  OR  OVER  THE 
FIRM  FIXED  PRICE 

IMPOSES  A MINIMUM  ADMINISTRATIVE  BURDEN  ON  THE  CONTRACTING 
PARTIES 


ret/e  am?  L£u<*rt/  ^enr/oA/f^/F 

APPLICATION:'; 

SUITABLE  FOR  USE  WHEN  DEFINITE  DESIGN  OR  PERFORMANCE  SPECIFICATIONS 
ARE  AVAIUBLE  AND  FAIR  PRICE  CAN  BE  ESTABLISHED  FROM  EITHER: 

- ADEQUATE  PRICE  COMPETITION 

- PRIOR  PURCHASE  HAVE  BEEN  MADE  OF  SAME  OR  SIMIUR  ITEMS 
SUPPORTED  BY  VALID  COST  OR  PRICING  DATA 

- COST  OR  PRICING  DATA  IS  AVAIUBLE  UPON  WHICH  REALISTIC 
ESTIMATES  OF  THE  PROBABLE  COSTS  OF  PERFORMANCE  CAN  BE  MADE 

- IDENTIFICATION  OF  THE  UNCERTAINTIES  OF  PERFORMANCE  AND 
ASSOCIATED  RISKS  WITH  ADEQUATE  MEANS  AVAIUBLE  TO  PRICE 
SAME 

• NORMALLY  USED  FOR  STANDARD  OR  MODIFIED  COMMERCIAL  ITEMS  OR  LOW 
TO  MEDIUM  TECHNOLOGY  ITEMS  WHERE  DEVELOPMENT  IS  COMPLETE  AND 
PRODUCTION  EXPERIENCE  IS  AVAIUBLE  TO  SUPPORT  PRICING. 
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FIXED  PRICE  CONTRACT  WITH  ESCALATION 


DESCRIPTION-. 

• PRICE  IS  SUBJECT  TO  UPWARD  AND/OR  DOWNWARD  REVISION  UPON  THE 
OCCURRENCE  OF  SPECIFICALLY  DEFINED  ECONOMIC  CONTINGENCIES 

- NORMALLY  RELATED  TO  LABOR  AND  MATERIAL  COST  WITH 
ADJUSTMENT  BASED  ON  FORMULA,  ECONOMIC  INDICES, 

OR  CHANGES  TO  PUBLISHED  PRICING 

- NORMALLY  INCLUDES  NOT- TO-EXCEED  CEILING  PRICE 


FIXED  PRICE  INCENTIVE  CONTRACTS 


^DESCRIPTION^ 

• CONTRACT  PROVIDES  FOR  A NEGOTIATED  TARGET  COST,  TARGET  PROFIT, 
TARGET  PRICE,  AND  A CEILING  PRICE  (BUT  NOT  A PROFIT  CEILING  OR 
FLOOR)  AND  A FORMULA  FOR  ESTABLISHING  FINAL  PROFIT  AND  PRICE 


INCENTIVE  FEE  STRUCTURE  CAN  PROVIDE  FOR  PROFIT  ADJUSTMENT 
BASED  ON  COST  PERFORMANCE  ONLY  OR  CAN  INCLUDE  ADDITIONAL 
ADJUSTMENT  FACTORS  FOR  SCHEDULE  AND  OPERATIONAL  PERFORMANCE 

FIRM  TARGETS 

SUCCESSIVE  TARGETS  (INITIAL  TARGET  FRIOR  TO  FIRM  TARGET  WITH 
FORMULA  ADJUSTMENT) 

NEGOTIATED  INTERIM  BILLING  PRICE  OR  PROGRESS  PAYMENTS  MAY 
SERVE  AS  BASIS  FOR  PAYMENT  WITH  EXTENDED  PERIOD  OF 
PERFORMANCE 


• USE  WHEN  FFP  IS  NOT  FEASIBLE  BUT  CIRCUMSTANCES  ARE  OF  SUCH  A NATURE  THAT 
ASSUMPTION  OF  A DEGREE  OF  COST  RESPONSIBILITY  PROVIDES  THE  CONTRACTOR 
WITH  A POSITIVE  PROFIT  INCENTIVE  FOR  EFFECTIVE  COST  AND  PERFORMANCE  CONTROL 

- HIGHER  RISK  TO  CONTRACTOR  THAN  CPIF  DUE  TO  CEILING  PRICE 

• REQUIRES  ADEQUATE  CONTRACTOR  ACCOUNTING  SYSTEM  FOR  TRACKING  AND  AUDIT 
OF  ALLOWABILITY  AND  ALLOCABILITY  OF  COSTS 

• SUCCESSIVE  TARGETS  ONLY  USED  WHEN  INITIAL  COST  AND  PRICING  DATA  IS 
INSUFFICIENT  TO  PERMIT  REALISTIC  FIRM  TARGETS 


FIXED  PRICE  REDETERMINABLE  CONTRACT 


|descrift;on:| 

• PROVIDES  FOR  A FIRM  FIXED  PRICE  FOR  INITIAL  PERIOD  OF  PERFORMANCE 
AND  FOR  PROSPECTIVE  PRICE  REDFTERMINATION  (UPWARD  OR  DOWNWARD) 
FOR  SUBSEQUENT  PERIODS  OF  PERFORMANCE 

- A SERIES  OF  FFP  CONTRACTS 

- NORMJ  LLY  INCLUDES  A CEILING  PRICE  FOR  THE  TOTAL  PERIOD  OF 
PERFORMANCE  SO  CONTRACTOR  ASSUMES  A MAJOR  PORTION  OF  RISK 

[ A.-FLICATIcTTj 

• WHERE  THE  FAIRNESS  OF  REASONABLENESS  OF  A FFP  IS  TIME  ORIENTED 
AND  CAN  ONLY  BE  ESTABUSHED  FOR  AN  INITIAL  PERIOD  BUT  NOT  FOR 
SUBSEQUENT  PERIODS 

• CONTRACTOR'S  ACCOUNTING  SYSTEM  MUST  BE  ADEQUATE  FOR  PRICE 
REDETERMINATION  PURPOSES 

• REQUIRES  TIMELY  PRICE  REDETERMINATION 


RETROACTIVE  FIXED  PRICE  DETERMINATION  CONTRACT 


COST,  NO  FEE  CONTRACT 


DESCRIPTION: 


• PROVIDE  FOR  THE  PAYMENT  TO  THE  CONTRACTOR  ALL  ALLOWABLE 
AND  ALLOCABLE  COSTS  INCURRED  IN  THE  PERFORMANCE  OF  THE 
CONTRACT  BUT  NO  FEE 


APPLICATION. 


• REQUIRES  ADEQUATE  CONTRACTOR  ACCOUNTING  SYSTEM  AND 
EXTENSIVE  SURVEILLANCE  OVER  SPENDING  TO  AVOID  INEFFICIENCY 
AND  WASTE 


AMOUNT  TO  BE  FILLED 
IN  BY  SELLER 


• NORMALLY  ONLY  USED  FOR  RESEARCH  AND  DEVELOPMENT  WORK 
WITH  NON-PROFIT  INSTITUTIONS  OR  ORGANIZATIONS 


DESCRIPTION: 


• PROVIDES  FOR  A CEILING  PRICE  AND  RETROACTIVE  PRICE  REDETERMINATION  AFTER 
COMPLETION  OF  THE  CONTRACT 

_ BASIS  FOR  PRICE  REDETERMINATION  SHOULD  BE  PRE-ESTABLISHED  AND  SHOULD 
NORMALLY  EMPHASIZE  MANAGEMENT  EFFECTIVENESS  AND  INGENUITY 
DURING  PERFORMANCE 


APPLICATION:| 


WHERE  FAIR  AND  REASONABLE  PRICE  CANNOT  BE  NEGOTIATED  FROM  OUTSET 
AND  OTHER  TYPES  OF  CONTRACTS  ARE  IMPRACTICABLE 

NORMALLY  SMALL  VALUE  (UNDER  $100,000)  AND  SHORT  PERIOD  OF  PERFORMANCE 
(LESS  THAN  SIX  MONTHS) 

CEILING  PRICE  SHOULD  IMPOSE  SOME  DEGREE  OF  RISK  ON  CONTRACTOR 

REQUIRES  ADEQUATE  ACCOUNTING  SYSTEM  FOR  PRICE  REDETERMINATION 
PURPOSES 

PRICE  REDETERMINATION  SHOULD  BE  TIMELY 
CANNOT  BE  COST-PLUS-A-PERCENTAGE-OF-COST 


DESCRIPTION 


APPLICATION 


• REQUIRES  ADEQUATE  ACCOUNTING  AND  SURVEILLANCE 


COST  SHARING  CONTRACT 


• A COST  REIMBURSEMENT  TYPE  CONTRACT  UNDER  WHICH  THE  CONTRACTOR 
RECEIVES  NO  FEE  AND  IS  REIMBURSED  ONLY  FOR  AN  AGREED  PORTION  OF 
HIS  ALLOWABLE  AND  ALLOCABLE  COSTS 


• JOINTLY  SPONSORED  PROJECT  WHERE  IT  IS  AGREED  THAT  THE  CONTRACTOR 
RECEIVES  A SUFFICIENT  COMMERCIAL  OR  OTHER  BENEFIT  THAT  HE  SHOULD 
ABSORB  A PORTION  OF  THE  COSTS  OF  PERFORMANCE 

- E.G.:  DEVELOPING  A TECHNOLOGICAL  CAPABILITY  THAT  HAS  A 

SIGNIFICANT  FUTURE  SALES  POTENTIAL 


COST  - PLUS  - INCENTIVE  FEE  CONTRACT 


A COST  REIMBURSEMENT  CONTRACT  WHERE  THERE  IS  INITIALLY  NEGOTIATED  A TARGET 
COST.  A TARGET  EEE,  A MINIMUM  AND  MAXIMUM  FEE,  AND  A FEE  ADJUSTMENT 
FORMULA  NORMALLY  BASED  UPON  ACTUAL  COST,  SCHEDULE,  AND  OPERATIONAL 
PERFORMANCE 

- CONTRACTOR  IS  PAID  ALL  a LLOWABLE  AND  ALLOCABLE  COSTS  I NCURRED 

IN  THE  PERFORMANCE  OF  THE  CONTRACT  AND  A FINAL  FEE  IN  ACCORDANCE 
WITH  THE  NEGOTIATED  TARGET  FEE  ADJUSTMENT  PROVISIONS  OF  THE 
CONTRACT 

- INTERIM  BILLING  FEE  IS  NEGOTIATED  AND  INCLUDED  IN  THE  CONTRACT 

THE  PROVISION  FOR  INCREASE  OR  DECREASE  IN  THE  FEE  (USING  TARGET  FEE  AS 
THE  BASELINE)  IS  DESIGNED  TO  PROVIDE  AN  INCENTIVE  FOR  MAXIMUM  EFFORT 
ON  THE  PART  OF  THE  CONTRACTOR  TO  MANAGE  THE  CONTRACT  PERFORMANCE 
EFFECTIVELY 


I A ' 'l,i'  * ~‘V1 

• USED  PRIMARILY  FOR  FEASIBLE  DEVELOPMENT  AND  TEST  TYPE  CONTRACTS  WHERE 
PERFORMANCE  PARAMETERS  ARE  DEFINED  AND  THERE  IS  A HIGH  PROBABILITY  THEY 
CAN  BE  ACHIEVED  AND  IT  IS  ADVISABLE  TO  PROVIDE  THE  CONTRACTOR  WITH  A 
VARIABLE  FEE  STRUCTURE  AS  AN  INCENTIVE  TO  EFFECTIVE  MANAGEMENT 

• IN  NEGOTIATING  FEE  ADJUSTMENT  PROVISIONS,  MUST  BALANCE  EMPHASIS  ON 
WEIGHT  GIVEN  TO  COST,  SCHEDULE,  AND  OPERATION  PERFORMANCE  FACTORS 
IN  ACCORDANCE  WITH  PROGRAM  OBJECTIVES 

NX  • REQUIRE  ADEQUATE  CONTRACTOR  ACCOUNTING  SYSTEM  BUT  tiSS  SURVEILLANCE 
\ THAN  CPFF  DUE  TO  INCENTIVE  STRUCTURE  MOTIVATION  FOR  CONTROL 


J 
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COST  - PLUS  - FIXED  FEE  CONTRACT 


• PROVIDES  FOR  Tl  lE  REIMBURSEMENT  TO  THE  CONTRACTOR  OF  ALL  ALLOWABLE  AND 
ALLOCABLE  COSTS  INCURRED  IN  THE  PERFORMANCE  OF  THE  CONTRACT  AND  A 
NEGOTIATED  FIXED  FEE  THAT  DOES  NOT  VARY  WITH  ACTUAL  COST  EXPERIENCE 
AGAINST  THE  CONTRACT 

- STATUTORY  FEE  LIMITATIONS: 

15%  OF  ESTIMATED  COST  FOR  EXPERIMENTAL,  RESEARCH,  AND 
DEVELOPMENT  CONTRACTS 

10%  OF  ESTIMATED  COST  FOR  PRODUCTION  CONTRACTS 

• MINIMAL  RISK  TO  CONTRACTOR  AND  MINIMUM  INCENTIVE  FOR  EFFECTIVE  MANAGE- 
MENT CONTROL  OF  COSTS 


UPfi  If  * ’ 
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NORMALLY  USED  IN  RESEARCH,  STUDY,  OR  DEVELOPMENT  CONTRACTS  INVOLVED  WITH 
ADVANCING  THE  STATE-OF-THE-ART  WHERE  SIGNIFICANT  UNKNOWNS  EXIST  WITH 
RESPECT  TO  THE  LEVEL  OF  EFFORT  REQUIRED  FOR  PERFORMANCE,  SOME  FEE  IS  DETER- 
MINED TO  IE  JUSTIFIED,  AND  A CPIF  CONTRACT  IS  DETERMINED  NOT  TO  IE  FEASIILE 

REQUIRES  ADEQUATE  CONTRACTOR  ACCOUNTING  SYSTEM  AND  EXTENSIVE  SURVEIL- 
LANCE OVER  SPENDING  TO  AVOID  INEFFICIENCY  AND  WASTE 


StU£K. 


COST  - PLUS  _ AWARD  FEED  CONTRACT 


CONTRACTOR’S  FEE  IS  DETERMINED  SUBJECTIVELY  BY  THE  BUYER  ON  THE  BASIS  OF 
PERIODIC  AFTER-THE-FACT  EVALUATION  OF  THE  CONTRACTOR'S  PERFORMANCE 
AGAINST  SPECIFIED  BROAD  CRITERIA  FOR  EACH  PERFORMANCE  PERIOD 


- FEE  CAN  BE  AWARD  ONLY  BUT  AWARD  FEATURE  IS  NORMALLY  INCLUDED 
WITHIN  A FIXED  OR  INCENTIVE  FEE  STRUCTURE 


- AWARD  FEE  DECISION  IS  UNILATERAL  AND  NOT  SUBJECT  TO  DISPUTES 
CLAUSE  PROCEDURE 


I 
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MAY  BE  USED  FOR  HARDWARE  DEVELOPMENT  AND  TEST 
CONTRACTS  UNDER  NORMALLY  CPFF  OR  CPIF  SITUATIONS 
WHERE  AN  AWARD  FEE  PROVISION  IS  CONSIDERED  AN 
APPROPRIATE  MOTIVATION  TO  MANAGEMENT  TOWARD 
THE  ACCOMPLISHMENT  OF  CONTRACT  OBJECTIVES 


ALSO  USED  FOR  BASE,  MISSION  SUPPORT,  AND 
COMPUTER  CENTER  OPERATION  AND  MAINTENANCE 
CONTRACTS 


TIME  AND  MATE Rl A 1/ LABOR  HOUR  CONTRACTS 


rDESCRIPTICN?. 

- 

• T&M  CONTRACTS  PROVIDE  FOR  THE  PROCUREMENT  OF  SUPPLIES  OR  SERVICES 
ON  THE  BASIS  OF  PAYMENT  FOR  DIRECT  LABOR  HOURS  AT  SPECIFIED  FIXED 
HOURLY  RATES  (WHICH  RATES  INCLUDE  DIRECT  AND  INDIRECT  LABOR,  OVER- 
HEAD, AND  PROFIT)  AND  MATERIALS  AT  COST 

- NORMALLY  INCLUDES  A CEILING  PRICE 


- DOES  NOT  PROVIDE  THE  CONTRACTOR  WITH  ANY  POSITIVE  PROFIT 
INCENTIVE  TO  CONTROL  LABOR  AND  MATERIAL  COST  EFFECTIVELY 


• LH  CONTRACTS  ARE  SIMILAR  TO  T&M  TYPE  CONTRACTS  AND  DIFFER  ONLY 
BECAUSE  THE  CONTRACTOR  IS  NOT  REQUIRED  TO  FURNISH  ANY  MATERIAL 


l APPLICATION-; 

• USED  ONLY  IN  THOSE  SITUATIONS  WHERE  IT  IS  NOT  POSSIBLE  AT  THE  TIME 
OF  PLACING  THE  CONTRACT  TO  ESTIMATE  THE  EXTENT  OR  DURATION  OF 
THE  EFFORT  OR  TO  ANTICIPATE  COSTS  WITH  ANY  SUBSTANTIAL  ACCURACY 

- E.G.:  REPAIR,  MAINTENANCE,  OR  OVERHAUL  WORK 
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THE  ROLE  OF  CONTRACTUAL  INCENTIVES 

Hr.  Patrick  R.  McGinnis 
Director  of  Contracts 
McDonnell  Douglas  Corp. 
Huntington  Beach,  CA 


"To  be  meaningful,  contract  incentives  must  moti- 
vate the  incentivized  party  to  apply  more  emphasis 
on  key  program  issues  than  he  would  have  applied 
absent  incentives."  Mr.  McGinnis  continued  this 
theme  by  showing  if  we  are  to  focus  attention  then 
the  candidate  elements  (cost,  schedule,  performance) 
and  the  candidate  events  must  be  identified  and 
prioritized. 

Three  primary  considerations  must  be  addressed: 

(1)  The  total  amount  of  potential  earnings 
to  be  tied  to  incentives  and  the  alloca- 
tion of  that  amount  across  incentive 
elements  and  events. 

(2)  Selected  incentive  events  must  be  examined 
to  assure  they  can't  inadvertently  contri- 
bute to  program  problems  (encourage 
unadvisable  technical  shortcuts  or  unreal- 
istic estimates). 

(3)  Incentive  flowdown  to  individuals. 

On  this  last  consideration,  Mr.  McGinnis  shared  an 
experience  of  McDonnell  Douglas  with  incentive  flow- 
down.  In  a recent  CPAF  contract  for  off-site  per- 
formance, the  company  set  up  an  arrangement  to  share 
60S  of  the  award  with  all  the  employees.  Mr. 
McGinnis  observed  that  in  this  situation,  each 
individual  could  accurately  identify  his  role  in 
attaining  the  award.  This  experience  invoked  a lot 
of  interested  discussion  from  the  floor.  The 
audience  was  skeptical  about  whether  this  arrange- 
ment would  work  in  other  than  an  off-site  contract. 
There  was  also  some  doubt  that  most  corporate 
management  would  settle  for  such  a low  total  profit 
on  a substantial  size  contract. 

On  the  subject  of  award  fees,  Mr.  McGinnis  favored 
their  increased  use  if  applied  within  the  following 
guidel ines : 

(1)  "The  Government  should  provide  broad 
guidelines  for  the  Incentive  Plan  and 
prioritize  them.  The  contractor  should 
be  allowed  to  propose  the  details  of  the 
plan. 

(2)  Both  parties  must  understand  the  total 
earnings  available. 

(3)  Subcontractor  and  individual  flowdown 
must  be  addressed. 

(4)  The  motivational  aspects  of  the  award 
fee  vis-a-vis  the  incentive  fee  should  be 
compared  and  evaluated. 

The  Value  of  Incentives 

• To  be  meaningful,  contract  Incentives  must 
motivate  the  incentivized  party  to  apply 
more  emphasis  on  key  program  issues  than 
he  would  have  applied  absent  incentives. 


• Assuming  the  prospect  of  more  (or  less) 
than  base  earnings  serves  to  focus  empha- 
sis, the  candidate  incentive  elements 
must  be  prioritized,  i.e.,  cost,  schedule, 
performance;  and  the  candidate  incentive 
events  within  the  schedule  and  perfor- 
mance elements  must  be  identified. 

• Having  selected  the  elements  and  events 
to  be  incentivized,  three  primary  con- 
siderations must  be  addressed: 

1)  The  total  amount  of  potential  earn- 
ings to  be  tied  to  incentives  and  the 
allocation  of  that  amount  across 
incentive  elements  and  events. 

2)  Selected  incentive  events  must  be 
examined  to  assure  they  can't 
inadvertently  contribute  to  program 
problems  (encourage  unadvisable 
technical  shortcuts  or  unrealistic 
estimates) . 

3)  Incentive  flowdown  to  individuals. 

The  latter  consideration  oust  be  viewed 
from  the  standpoint  of  whether  maximum 
motivation  can  be  attained  by  increasing 
the  profit  potential  of  the  contracting 
entity  or  by  increasing  the  earnings  of 
individuals  working  on  the  contract  by 
means  of  a specific  flow  through  of 
incentive  dollars. 

• The  preceding  comments  relate  to  objective 
incentives  (bonus  or  penalty  relates  to 
meeting  a specific  and  predetermined  cost, 
schedule  or  performance  goal).  Use  of 
subjective  (award  fee)  incentives  have 
increased  in  recent  years.  The  motiva- 
tional aspects  of  the  award  fee  vis  a vis 
the  incentive  fee  approach  should  be 
compared  and  evaluated. 


ARE  INCENTIVES  EFFECTIVE? 

Mr.  Harold  E.  Sharp 
SAMSO , Chief  of  Pricing 

Mr.  Sharp  started  his  presentation  by  providing 
three  questions  which  must  be  asked  whenever  one  is 
considering  the  use  of  performance  incentives,  i.e., 
are  the  incentives:  (1)  desirable  (2)  attainable 
(3)  measurable.  Mr.  Sharp  suggested  that  when 
SAMSO  and  the  Air  Force  look  at  Mission  Assurance, 
they  look  at  the  Total  Program.  He  identified  this 
as  a problem  from  the  government's  viewpoint  since 
we  do  not  contract  for  a total  program.  The  launch 
vehicle,  spacecraft,  launch  services,  and  orbital 
support,  though  all  a part  of  the  total  program  and 
required  for  mission  success,  each  is  contracted 
for  separately  and  consequently,  the  incentive 
environment  is  splintered.  Mr.  Sharp  suggests  that 
the  incentive  plan  would  be  different  in  a competi- 
tive environment  than  in  a sole  source  environment. 
He  believes  that  when  the  two  sides  to  a negotia- 
tion have  a large  difference  in  total  price,  the 
performance  incentive  becomes  a mechanism  to  give 
both  sides  what  they  want. 


Mr.  Sharp  recommends  use  of  the  Predetermined 
Incentive  Plan.  In  the  ideal  situation,  the  buyer 
and  seller  would  agree  on  the  details  of  the  incen- 
tive plan  without  addressing  dollars  prior  to  the 
start  of  negotiations. 

After  these  remarks,  Mr.  Sharp  shared  the  SAMSO 
incentive  experience  and  statistics  with  the 
audience. 

THE  VALUE  OF  INCENTIVES  AS  A MOTIVATION 
FOR  HIGH  MISSION  ASSURANCE 


1 . Kinds  of  Motivation 

a.  Company  Image 

b.  Long  range  business 

c.  Pride  of  the  individual  and/or  team 

d.  Profit  (immediate  contract) 

2.  Origin  of  the  Incentive  Plan 

a.  Competitive  Environment  - dictated  in 
RFP,  with  limited  input  by  contractor. 

b.  Sole  Source  Environment 


b.  Steps  functions 

c.  Linear 

d.  Non-linear 

4.  Interdependency 

a.  Implicit 

b.  Explicit 

5.  Trade-Off  Analysis 

a.  Three  dimensional  trade-offs 

(1)  Cost  share  line  and  limits 
(ceiling  price,  max/mi n fee) 

(2)  Performance  par,  limits,  proba- 
bility and  distribution 

(3)  Timing  and  future  value  of 
payment 

b.  Graphic  techniques 

(1)  Isofee  curves 

(2)  Nomographs 

(3)  Incentive  cost/profit  graphs 
and  overlays 

c.  Problems  in  creating  "ideal"  trade- 
offs 

(1 ) "Ideal"  isofee  curve 

(2)  Combining  linear  cost  and  per- 
formance incentives 

(3)  Non-linear  performance 
incentives 

6.  SAMSO  Incentive  Plans 

a.  Satellite 

b.  Boosters 

c.  I CBM 

d.  Other 


(1)  Negotiation  technique  to  sweeten 
the  "deal"  by  increased  profit 
potential  for  lower  costs. 

(2)  Predetermined  incentive  plan 
based  on  Program  Offices  desires. 
Fine  tuned  during  negotiation. 

(3)  Follow-on  buys  - similar  to 
previous  contract. 


3. 


Structuring  Performance  Incentives 
a.  Go-no-go 


SAMSO  SATELLITE  PERFORMANCE  INCENTIVES 


PROGRAM/CONTRACT 

SHARE/ 

? INCENTIVES 

MONTHLY 

DOLLAR 

PAR 

TOTAL 

TIME  OF 

DESCRIPTION 

TYPE 

PERCENT 

SCHD 

PERF 

INCENTIVES 

YRS 

YEARS 

PAYMENT 

Motive 

POSITIVE 

AMT 

MOS 

AMT 

MOS 

TsoooT- 

TToool- 

1.  DSCS  II 

FPIF 

80/20 

+1.1 

+ 15/ 

91.7 

12 

0 

12 

1-2 

5 

Bimonthly 

6 Satellites 

T.P  11.0? 

-10 

91.7 

12 

after  2nd 

C.P  125? 

22.9 

24 

year 

2.  OCCS  II 

FPIF 

80/20 

+ 1.2 

+ 15/ 

131.6 

12 

131.6 

24 

1 

5 

Bimonthly 

4 Satellites 

T.P  11? 

-10 

65.8 

24 

after  1st 

C.P  120? 

year 

3.  FLTSATCOM 

FPIF 

80/20 

+6.4 

+13.4 

-0- 

91.5 

36 

0 

3 

Pre-pay 

2 Satellites 

T.P  6.1? 
C.P  125? 

after  c/o 

4.  FLTSATCOM 

FPIF 

80/20 

-0- 

+6.7/ 

75.0 

24 

75.0 

36 

2 

5 

Pre-pay 

1 Satellite 

T.P  12? 

C.P  120? 

-4.5 

after  c/o 

5.  DCSC  III 

FPIF 

80/20 

-0- 

+27/ 

306.3 

12 

91.9 

120 

0-1 

10 

Bimonthly 

2 Satellites 

T.P  4.1? 

-9 

Overlap 

C.P  124? 

6.  NATO  III  (2) 

FPIF 

60/40 

-3 

+8/ 

61.4 

36 

81.9 

24 

3 

5 

Bimonthly 

1 Satellite 

T.P  12? 

-9 

after  3rd 

C.P  125? 

year 

7.  NAVSTAR  (2) 

FPIF-AF 

80/20 

-0- 

+4.3/ 

11.4 

36 

30.6 

36 

0-3 

3 

Annually 

3 Satellites 

T.P  6.2? 

-1.4 

Overlap 

C.P  126.2? 
A. F.  2.5? 

8.  DMSP 

FPIF 

* 

+ 1.5/ 

+7.5 

-0- 

98.6 

2.5 

1.5Mo 

2 

Milestone 

2 Satellites 

T.P  11.5? 

-0.3 

30.6 

8.2 

C.P  128?  40.9  6.1 

41.2  6.1 

*95/5  T.C.  +8? 

90/10  T.C.  +(9-13?) 

75/25  T.C.  +14? 

75/25  Underrun 

NOTE: 

(1)  Amounts  shown  are  for  basic  negotiated  contract  values 

(2)  Competitive  awards 


J 


SAMSO  PERFORMANCE  INCENTIVES  - OTHER  (1) 


PROGRAM  CONTRACT 

SHARE/ 

DESCRIPTION 

TYPE 

PERCENT 

1.  TITAN  - TRANSTAGE 
INJECTOR 

CPIF 

85/15  Over 
90/10  Under 
T.F.  8.3^ 
Min.  1% 

Max.  15% 

2.  TITAN-LAUNCH  SERVICES 

FPIF 

T.P.  11.7% 
C.P.  118% 

3.  TITAN-LAUNCH  SERVICES 

FPIF 

70/30  Over 
85/15  Under 
T.P.  11.4% 
C.P.  120% 

4.  MINUTEMAN-MK  12A 
PRODUCTION 

CPIF 

85/15 

T.F.  7% 

Min.  4% 

Max.  10% 

5.  MINUTEMAN-GYRO 

ACCELEROMETERS  (PIGA) 

FY  76 

FPIF 

75/25 

T.P.  10.8% 
C.P.  117% 

6.  MINUTEMAN-GYRO 

Accelerometers  (PIGA) 

FY  77 

FPIF 

75/25 

T.P.  10.2% 
C.P.  115.2% 

NOTE: 

(1)  Amounts  shown  are  for  basic  negotiated  contract  values. 


INCENTIVES 


+$267K/-$178K  Improved  Specific  Impulse 
Performance  +6/-4%,  Schedule  +1.2% 


No  Fail 

1 Fail 

2+  Fail 

Under 

60/40 

75/25 

100/0 

Over 

80/20 

80/20 

80/20 

No  Fail 

1 Fail 

3+  Fail 

+S30K* 

+$20K* 

-$50K** 

* Per  Successful  Vehicle 
**  3rd  and  Subsequent  Vehicles 

Production  Hour  Incentive 

+ 1.8* 


MTBF  1000  Hour:  Increments  +14.5% 
Min  45,000  Hrs , Par  75,000  Hrs. 

Max  100,000  Hrs 

MTBF  1000  Hr,  Increments  +12.7% 

Min  55,000  Hrs,  Par  100,000  Hrs. 
Max  224,000  Hrs 
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ARE  INCENTIVES  USEFUL  TO  ACHIEVE  MISSION  SUCCESS? 

Mr.  Grant  L.  Hansen 
President 

System  Development  Corporation 


Mr.  Hansen's  remarks  were  of  particular  interest  to 
the  audience  in  light  of  his  past  experience  as 
Vice  President  for  Space  of  the  Convair  Division  of 
General  Dynamics  and  as  the  former  Under  Secretary 
of  Defense  for  Research  and  Development.  The  com- 
plete text  of  Mr.  Hansen's  remarks  follow  and  were 
the  stimulus  for  much  discussion  during  the  remain- 
der of  the  workshop. 

Colonel  JuVette,  fellow  panelists,  ladies  and 
gentlemen  - it's  a pleasure  for  me  to  be  here. 

Like  baseball,  hot  dogs,  apple  pie,  and  Chevrolet, 
mission  success  is  all-American  goodness  and  we  are 
all  for  it.  I hope  this  series  of  workshops  can 
help  us  to  get  mere  of  it.  I am  honored  to  be  a 
participant  in  this  panel  which  is  considering  the 
benefit  of  contractual  performance  incentives  in 
improving  mission  success  probability  through  con- 
tractor management  motivation.  It  is  a worthy  sub- 
ject and  I do  have  some  ideas  about  it,  which  I am 
pleased  to  have  this  opportunity  to  express. 

I was  privileged  to  have  been  a participant  in  one 
of  General  Morgan's  sessions  with  industry  repre- 
sentatives to  discuss  what  our  Air  Force-Industry 
team  can  do  to  enhance  the  chances  of  achieving  the 
100.  success  record  we  all  want.  I commend  those 
who  have  organized  these  meetings  so  that  we  can 
all  expand  on  what  has  already  been  done. 

Since  1 originally  agreed  to  participate  in  this 
panel,  I have  retired  from  General  Dynamics  and 
joined  System  Development  Corporation  of  Santa 
Monica.  Therefore,  I can  no  longer  speak  for 
General  Dynamics  and  I'm  too  new  at  SDC  to  speak 
for  them,  so  I'll  just  speak  cor  myself.  I do  have 
inputs  from  both  companies  and  I will  review  the 
use  of  contractual  performance  incentives  at 
Convair,  talk  about  why  launch  vehicles  are  differ- 
ent from  spacecraft  and  take  the  position  that  for 
today's  launch  vehicles,  at  least,  there  are  better 
ways  for  the  Air  Force  to  spend  their  mission  assur- 
ance money  than  by  putting  it  into  contractual  per- 
formance Incentives. 

In  the  1 950 ' s and  early  1 960 ' s we  were  almost  more 
surprised  when  a launch  vehicle  went  right  than 
when  it  went  wrong.  The  first  Atlas  ICBM  launch 
was  June  11,  1957  and  it  was  unsuccessful.  During 
the  next  two  years  there  were  twenty-five  Atlas 
launches  and  fifteen  were  unsuccessful.  During  the 
next  five  years  there  were  another  hundred  Atlas 
launches  with  only  sixty-five  successful  ones.  The 
overall  reliability  to  mid  1964  was  only  60T. 
Everyone  tried  hard,  but  the  machines  were  very 
complex  and  Immature.  Murphy's  law  was  having  its 
heyday.  Everyone  was  concerned  about  low  space 
launch  reliability  and  struggling  to  do  something 
to  Improve  it. 

Our  first  management  financial  incentive  was  when 
John  Rubel , who  was  then  in  DOD,  made  a $10  bet 
with  Jim  Dempsey,  who  was  then  General  Manager  of 
Convair,  that  we  couldn't  get  seven  or  more  succes- 
ses out  of  the  next  ten  Atlas  space  launches.  Most 
of  us  who  were  working  on  the  program  shared  Rubel 's 


belief  that  he  could  easily  win  that  bet.  Jim 
Dempsey  had  a very  fancy  plaque  made,  with  a space 
identifying  each  launch  and  indicating  success  or 
failure.  He  hung  the  plaque  prominently  in  his 
office  so  that  it  couldn't  be  missed  by  any  of  us 
or  any  visitor  who  went  into  his  office. 

No  one  can  ever  prove  whether  that  bet  helped,  or 
whether  the  configuration  had  simply  matured  to  the 
point  that  it  was  ready  to  start  working  better 
anyway,  but  Jim  Dempsey  won  that  bet  going  away! 

Nine  of  the  ten  launches  were  successful. 

A special  type  of  success  motivation  was  put  into 
the  Atlas  program  when  it  was  designated  as  the 
launch  vehicle  for  the  Mercury  Astronauts.  Those 
astronauts  visited  our  plant  and  talked  with  all  of 
our  people,  so  that  they  had  a personal  identifica- 
tion with  the  mission  and  understood  that  every- 
thing had  to  be  done  just  right  to  preserve  the 
safety  of  those  great  American  heros. 

When  John  Glenn  was  launched,  I thought  he  must  be 
the  most  courageous,  or  foolhardy,  person  who  had 
ever  lived.  I felt  that  if  he  knew  as  well  as  I 
did  how  many  things  were  capable  of  going  wrong  he 
would  never  have  agreed  to  go.  I guess  he  probably 
did  know,  but  I just  had  no  idea  what  a man  might 
go  through  in  order  to  get  elected  to  the  Senate! 

The  Mercury  Program,  as  we  all  know,  was  a 100" 
success.  When  it  was  completed,  the  Air  Force, 

NASA,  Aerospace  Corporation,  Genera!  Dynamics  and 
the  Atlas  associate  contractors  conducted  a three 
day  workshop  so  that  the  Industry  and  all  who  were 
interested  could  hear  about  all  the  things  which 
were  done  to  deliver  that  100T  success  record. 
Contractual  performance  incentives  still  had  not 
been  used  in  the  Atlas  Program,  but  our  confidence 
in  our  ability  to  deliver  success  was  beginning  to 
be  quite  high.  All  of  the  special  care  features  of 
the  Mercury  launch  vehicle  program  were  incorpor- 
ated Into  the  Atlas  Space  Launch  Vehicle  program 
on  a routine  basis. 

Another  very  special  kind  of  success  motivation 
technique  was  used  by  NASA  and  the  Congress  during 
the  early  days  of  the  Centaur  Program.  The  first 
launch  was  May  8,  1962.  Ten  days  later.  Congres- 
sional hearings  were  started  to  review  that  failure 
and  the  Centaur  program  in  general.  Within  a few 
more  days  we  had  a full  scale  GAO  investigation 
going,  which  lasted  for  several  months.  NASA 
transferred  Centaur  responsibility  from  the 
Marshall  Space  Flight  Center  at  Huntsville  to  the 
Lewis  Research  Center  in  Cleveland  and  gave  us  one 
more  chance.  Congress  had  promised  to  watch  the 
program  very  carefully. 

Our  next  launch  was  27  November  1963  and  we  all 
knew  that  launch  had  to  be  a success  or  our  program 
would  be  cancelled.  Reporter  Bill  Hines  of  the 
Chicago  Sun  had  already  prepared  an  article  assuming 
failure  and  dubbing  Centaur  as  a "Thanksgiving 
Turkey”.  Fortunately,  that  flight  was  a big  success. 
The  only  thing  which  dampened  our  jubilation  was 
that  the  country  was  in  mourning  for  President 
Kennedy's  death  and  we  were  not  permitted  to  have  a 
launch  success  party. 

For  the  next  several  launches,  we  were  still  hanging 
on  the  ropes,  very  fearful  that  any  launch  failure 
would  trigger  a new  round  of  Congressional  and  GAO 


investigations,  with  the  end  result  of  program  can- 
cellation. We  all  carried  a card  which  said  "HOPE". 
That  acronym  stood  for  "Help  Our  Program  Endure". 

We  were  on  a CPFF  contract,  but  we  just  couldn't 
have  been  more  motivated.  After  each  successful 
launch  we  fully  appreciated  Winston  Churchill's 
words  when  he  said  "There  is  nothing  more  exhilarat- 
ing than  to  be  shot  at  and  missed;" 

We  were  especially  nervous  about  our  ability  to 
successfully  support  the  Surveyor  program  for  soft 
landings  on  the  moon.  We  understood  the  importance 
of  Surveyor  to  readiness  for  the  Apollo  manned 
landings  and  were  keenly  motivated  for  success  but 
trembling  with  the  knowledge  of  how  many  things 
could  go  wrong.  The  100"  success  record  on  launch- 
ing of  seven  Surveyors  established  our  confidence 
that  the  Atlas-Centaur  launch  vehicle  combination 
could  be  a reliable  work  horse  of  the  new  space 
age,  and  it  has  been. 

Of  all  Centaur  launches,  38  out  of  41  have  been 
successful.  Since  the  D Model  reliability  improve- 
ments, 20  out  of  20  have  been  successful,  with  2 
no- trials  because  of  booster  failures.  Special 
attention  was  given  to  the  Titan  Centaur  and  it  has 
had  1002  success  for  6 operational  launches.  After 
the  "Preparedness  Program"  of  hardware  and  proce- 
dure changes,  the  E and  F Models  of  Atlas  have  had 
52  out  of  57  launches  successful  or  912  and  the 
SLV-3  Atlas  has  had  94  successes  out  of  99,  or  95%. 
Anything  less  than  100%  is  unacceptable  - that's 
why  we're  here.  The  person  whose  mission  just 
failed  doesn't  care  about  the  53  straight  successes 
you  just  completed  prior  to  his  failure. 

Convair's  first  performance  incentive  contract  was 
negotiated  with  the  Air  Force  in  1962.  It  was  for 
the  development,  production  and  launch  of  Atlas 
Standard  Space  Launch  Vehicles.  Many  reliability 
improvements  were  included,  in  addition  to  the 
special  care  features  which  had  been  learned  from 
the  Mercury  program.  The  Mercury  program  had  used 
the  basic  Model  0 version  of  the  Atlas  ICBM  vehi- 
cle, with  some  changes  for  mission  adaptation, 
safety  and  reliability.  The  new  Atlas,  SLV-3,  was 
to  be  the  best  and  most  reliable  vehicle  we  knew 
how  to  build. 

To  the  best  of  my  knowledge,  the  Atlas  SLV-3  con- 
tract was  the  first  launch  vehicle  contract  with 
contractual  launch  and  mission  performance  incen- 
tives. The  actual  performance  formula  was  some- 
what complicated,  but  essentially  it  provided  plus 
or  minus  incentive  fee  for  variations  from  a stip- 
ulated target  reliability  performance  goal.  The 
maximum  performance  incentive  fee  was  achieved  at 
90%  reliability.  We  did  earn  the  maximum  fee.  In 
my  den  at  home,  I have  a nicely  framed  picture  of 
Col.  Leo  Sullivan,  Atlas  SP0  Director  at  the  time, 
presenting  me  with  a check  for  nearly  $4  million. 

We  had  achieved  a reliability  of  94.3%  for  the  35 
vehicles  under  that  contract. 

Because  that  vehicle  series  was  redesigned  for 
reliability,  because  the  configuration  and  manufac- 
turing procedures  were  maturing,  because  the  crew 
was  gaining  experience  and  confidence  and  because 
we  had  a new  contractual  performance  Incentive,  it 
is  impossible  to  separate  the  variables  and  inde- 
pendently assess  the  value  of  the  contractual  per- 
formance incentive  fee.  Because  it  was  the  first  of 
its  kind  for  Convair  and  because  a lot  of  flexibil- 
ity for  contractor  action  existed  (most  of  which 


has  since  disappeared),  most  people  would  agree  that 
senior  management  attention  was  drawn  to  the  pro- 
gram more  thoroughly  by  the  incentive  fee  prospects. 
This  may  have  resulted  in  a better  reliability 
record  than  we  would  otherwise  have  had  - a point 
which  can't  be  proven  or  disproven.  Nature  has  a 
way  of  not  revealing  the  results  of  unused 
alternatives. 

I do  know  that  I really  enjoyed  receiving  the  per- 
formance fee  check,  I prize  the  picture  of  it  being 
presented  to  me,  and  I know  that  it  made  our  Con- 
troller and  my  bosses  very  happy. 

The  follow-on  contract  for  Atlas  SLV-3  launch  vehi- 
cles was  negotiated  in  1965  and  provided  fee  pen- 
alties for  anything  less  than  100%  success.  The 
penalty  was  a pre-established  amount  for  each  vehi- 
cle failure.  There  was  considerable  opposition  to 
this  arrangement  within  General  Dynamics  - the 
argument  being  that  negatives  were  not  as  effective 
motivators  as  positives.  It's  the  old  stick  versus 
carrot  motivation  argument  which  I don't  think  any- 
one has  ever  won,  but  a consensus  view  is  that  some 
of  each  is  probably  better  than  either  alone. 

As  a compromise  with  Convair,  the  Air  Force  finally 
agreed  to  a special  bonus  of  about  1%  fee  for  100% 
success.  The  penalty  for  each  failure  remained. 

That  contract  was  completed  in  1968,  with  a relia- 
bility record  of  100%  for  twenty-eight  vehicles. 
Convair  received  the  special  bonus  fee.  Since  that 
time,  the  government  has  shown  no  further  interest 
in  performance  incentives  for  Atlas  SLV-3  space 
launch  vehicle  contracts.  Also,  General  Dynamics 
has  not  pushed  for  this  feature  in  follow-on  con- 
tracts. I'll  say  more  about  this  later,  but  1 
would  first  like  to  round  out  the  historical  per- 
spective on  contract  performance  incentives  at 
Convair. 

In  the  late  '60s  and  early  '70s  Convair  had  two 
performance  incentivized  contracts  for  modification, 
Aircraft  Inspection  and  launch  of  E and  F Model 
Atlas  ICBM-s  which  were  left  over  from  the  deacti- 
vation of  the  Atlas  Weapon  System.  These  "wheat 
fields  birds"  have  been  and  are  being  used  for  space 
launches  from  Vandenberg  Air  Force  Base  as  well  as 
for  re-entry  vehicle  testing  and  as  targets  for  ABM 
testing  in  the  South  Pacific. 

Some  definition  of  terms  is  relevant  to  understand- 
ing the  results  of  these  contracts.  A success  is 
defined  as  the  payload  completing  its  flight  objec- 
tives or  the  Atlas  delivering  the  payload  to  the 
end  conditions  specified  by  the  Air  Force  for  each 
flight.  A failure  is  when  the  payload  does  not  meet 
its  flight  objectives  and  the  cause  is  attributable 
to  Convair  equipment  or  services.  A "No-Trial"  is 
defined  in  four  ways: 


1.  The  flight  is  terminated  or  the  payload 
mission  objectives  are  not  met  for  reasons 
outside  of  Convair's  area  of  direct 
responsibility,  or 

2.  Determination  of  responsibility  for  fail- 
ure cannot  be  made,  or 

3.  The  government  directs  a launch  against 
the  recommendation  of  Convair  management, 
or 
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4.  The  vehicle  was  not  launched  within  a 

prescribed  calendar  time  after  production 
delivery. 

The  actual  results  may  help  explain  why  the  Air 
Force  has  shown  no  further  interest  in  performance 
incentives  for  follow-on  launch  vehicle  contracts. 
Those  contracts  included  thirteen  vehicles  to  be 
launched  within  a time  period  ending  ten  months 
after  the  0D250  on  the  thirteenth  launch  vehicle. 
Only  eleven  vehicles  were  launched  within  that  time 
period  and  eight  of  the  eleven  were  no-trials.  The 
contract  specified  a minimum  of  nine  vehicles  to  be 
evaluated,  so  under  the  terms  of  the  contract  a 
performance  fee  of  9/13  of  the  maximum  incentive 
fee  had  to  be  paid. 

Convair's  only  other  experience  with  performance 
incentives  contracting  in  the  space  business  is  with 
the  NASA  contracts  where  award  fee  is  extensively 
used.  There  is  a target  fee,  typically  about  6 
to  7%,  and  an  award  fee,  typically  + a couple  of 
percent,  available  for  subjective  judgment  by  NASA 
of  contractor  performance. 

While  flight  success  is  an  important  consideration 
in  award  fee  determination,  there  are  many  others 
and  the  direct  linkage  with  flight  success  is 
obscure. 

Having  now  covered  the  historical  perspective  and 
all  of  the  contract  performance  incentive  experi- 
ence for  launch  vehicles  at  Convair,  I would  like 
to  make  these  eight  personal  observations: 

1.  As  compared  to  spacecraft  contracts, 
where  there  are  both  degrees  of  perfor- 
mance and  length  of  performance  that  can 
be  “valuated,  launch  vehicle  performance 
is  strictly  a go-no-go  situation. 

Because  of  the  relatively  small  quanti- 
ties that  are  procured  under  any  contract, 
it  is  not  feasible  to  use  a percentage  of 
total  performance  as  a goal;  hence,  as  we 
have  seen  in  our  experience,  the  tendency 
to  move  toward  fixed  bonus/penalty 
arrangements.  This  results  in  the  unde- 
sirable situation  from  the  Government 
standpoint  that  they  may  have  to  pay  a 
large  bonus  to  a launch  vehicle  contrac- 
tor even  though  the  mission  was  a failure 
because  of  upperstage  or  spacecraft  fail- 
ure. Though  I have  no  direct  evidence, 

I have  a feeling  this  was  one  of  the 
reasons  why  the  Air  Force  dropped  the 
incentives  on  the  SLV  program. 

2.  In  Convair's  case,  and  I suspect  for 
most  launch  vehicle  contractors,  there  is 
no  contractor  total  systems  responsibil- 
ity; i.e.,  engines  and  guidance  systems 
are  provided  as  GFE.  Because  of  this 
there  are  situations  wherein  the  launch 
vehicle  contractor  is  denied  a bonus  pay- 
ment he  otherwise  would  have  earned 
because  of  a failure  of  GFE.  The  Govern- 
ment has  recognized  this  aspect  of  the 
problem  and  has  in  effect  tried  to  pres- 
sure launch  vehicle  contractors  to  assume 
responsibility  for  the  GFE  systems  even 
though  they  had  no  contractual  arrange- 
ment, nor  any  real  control  over  the  asso- 
ciate contractor.  Additionally,  this 
divided  responsibility  makes  it  difficult 


to  determine  the  responsibility  for 
failure. 

3.  Because  of  launch  schedule  slippages,  the 
launches  sometimes  take  place  months  or 
even  years  after  the  vehicle  is  manufac- 
tured. Since  the  basic  design  for  most 
launch  vehicles  was  done  years  ago,  what 
is  really  being  incentivized  is  quality 
and  workmanship  rather  than  design  reli- 
ability. Incentives  paid  years  after  the 
work  has  been  performed  are  not  in  my 
opinion  very  effective  motivators.  It  may 
well  be  that  the  people  who  actually  did 
the  work  are  long  gone  or  on  a different 
project  by  the  time  the  incentives  are 
paid.  This  is  further  complicated  by  the 
fact  that  quite  often  the  vehicles  are 
launched  on  a different  contract,  and 
there  is  in  effect  a divided 
responsibility. 

4.  I understand  most  spacecraft  contractors 
are  in  favor  of  incentives.  Having  some 
knowledge  of  the  special  incentive  type 
programs  used  to  pay  them,  I can  under- 
stand at  least  one  reason  why.  In  the 
ones  with  which  I am  familiar,  aJJ  fee  was 
ultimately  earned  as  incentive  fee,  but  in 
fact  the  contractor  normally  drew  down  a 
healthy  advance  payment  of  fee  either  on 

a cost  reimbursement  basis  or  upon  com- 
pletion of  milestones.  He  then  earned  fee 
against  that  payment  if  the  spacecraft 
performed,  and  could  earn,  assuming  100% 
success,  15%  of  the  negotiated  cost.  If 
spacecraft  performance  is  poor,  the  con- 
tractor may  ultimately  have  to  pay  back 
some  or  all  of  the  previously  collected 
fee.  Given  the  development  time  for 
spacecraft,  this  method  even  under  the 
worst  circumstances  assures  him  the  use  of 
Government  money  for  a period  of  years. 

In  Convair's  launch  vehicle  contracts 
incentive  fee  has  never  been  a major 
component  of  the  earned  fee.  Given  the 
lack  of  total  systems  responsibility, 
they  would  be  reluctant  to  accept  a con- 
tract which  did  not  provide  a healthy 
minimum  fee.  Obviously  they  could  not  be 
in  a position  of  being  deprived  of  earn- 
ings because  of  an  associate  contractor 
failure.  Under  these  circumstances,  I am 
not  sure  that  the  Incentive  awards  for 
launch  vehicle  performance  can  ever  be 
made  large  enough  to  be  significant 
motivators. 


5.  Having  had  an  opportunity  to  participate 
in  Convair's  operation  under  both  incen- 
tive and  unincentivized  contracts,  I am  of 
the  opinion  that  there  was  essentially  no 
difference  in  our  approach  to  management 
or  motivation.  The  real  Incentive  to 
successful  performance  on  launch  vehicle 
contracts  is  the  avoidance  of  the  pain, 
turmoil  and  loss  of  image  that  goes  auto- 
matically with  a vehicle  failure.  Our 
ultimate  goal  has  always  been  to  sell 
more  launch  vehicles,  and  failures  don't 
help  the  sales  curve. 

Early  in  the  Convair  launch  vehicle  pro- 
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gram  we  put  success/failure  record  boards 
in  the  program  management,  engineering, 
production  and  launching  areas.  After 
each  launch  a posted  addition  to  the 
"track  record"  is  made  showing  the  results 
of  that  last  launch.  These  launch  reli- 
ability record  boards  are  watched  with 
great  Interest  by  all  concerned  and  I 
believe  they  have  been  a motivating  force. 

6.  The  primary  motivation  is  pride.  Each 
individual  involved  takes  pride  in  suc- 
cess and  feels  very  badly  about  failure. 
This  is  the  best  possible  motivation 
because  it  is  well  understood  that  when 
the  total  mission  is  successful  there  is 
great  pride  for  everyone.  If  the  mission 
is  unsuccessful,  there  is  little  consola- 
tion in  the  fact  that  "I  did  my  job 
right".  This  is  a great  motivator  to 
help  others,  in  addition  to  doing  one's 
own  job  right. 

7.  Having  been  through  many  failure  analyses 
and  reviews  it  is  a well  known  fact  that 
after  a failure,  the  people  involved  work 
day  and  night  and  weekends  for  weeks 
diagnosing  it  all,  determining  the  cor- 
rective action,  creating  the  reports  and 
attending  all  those  meetings.  Family  and 
other  personal  obligations  must  be  tem- 
porarily set  aside  - typically  for  a few 
weeks.  We've  tried  success  and  we've 
tried  failure  - success  is  infinitely 
better1.  I don't  think  individuals  can 
have  any  stronger  motivation  to  help 
produce  success. 

8.  Launch  vehicle  configurations  and  proce- 
dures are  very  mature  and  the  reliability 
data  base  is  large.  The  last  Atlas  launch 
was  number  439.  Experience  has  taught  us 
many  ways  to  Improve  reliability.  Most 
of  these  changes  have  been  Incorporated 
into  the  hardware,  the  software  or  the 
procedures.  However,  some  have  not  - 
because  of  lack  of  available  funds,  lack 
of  agreement  that  the  change  should  be 
made,  or  lack  of  feeling  that  the  number 
of  remaining  expendable  launches  can 
justify  a reliability  Improvement  change. 

A recent  consensus  of  Convair  senior  technical  and 
management  people  was  that  if  the  Air  Force  has 
additional  money  for  improvement  of  launch  mission 
assurance,  it  should  be  spent  on  recommended 
improvements  to  the  hardware  rather  than  on  con- 
tractual performance  incentive  fees.  However,  I 
doubt  if  the  financial  types  or  the  shareholders 
would  agree.  They  like  that  extra  profit  from 
Incentive  fees. 

For  my  own  part,  my  personal  opinion  is  that 
contractual  incentivizing  of  launch  success  will  not 
Improve  the  result.  The  Air  Force  Is  already  get- 
ting a very  high  level  of  motivation  and  dedication 
from  the  people  in  its  highly  developed  and  mature 
launch  vehicle  programs. 
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DO  MANAGEMENT  INCENTIVES  REALLY  WORK? 

Mr.  Robert  J.  Ingersoll 
Contracts  Mgr.  - MX  Program 
The  Boeing  Company 


During  Mr.  Ingersoll 's  presentation,  the  following 
points  were  discussed: 

1.  Question  from  the  audience:  "Is  manage- 
ment motivated  c"  'erently  through  objec- 
tive (performance)  incentives  and 
subjective  (award  fee)  incentives?" 

Mr.  Ingersoll:  "Only  a few  know  when  100%  per- 
formance incentive  is  earned,  but  everyone  knows 
when  100%  Award  fee  is  granted.  The  Award  fee  gets 
much  more  publicity." 

2.  Comment  from  the  floor  on  the  use  of 
Award  Fees: 

a.  Should  be  evaluated  3-4  times  per 
year. 

b.  Evaluation  stimulates  corrective 
action. 

c.  Evaluation  criteria  are  key;  i.e., 
they  must  be  clear  and  directed 
toward  desired  objectives. 

d.  There  should  be  an  opportunity  for 
the  contractor  to  defend  his  award 
claim. 

3.  Recommendation  from  the  floor  was  to 
somehow  merge  the  Design-To-Cost  (QTC) 
and  Value  Engineering  (VE)  concepts  in 
the  new  Defense  Acquisition  Regulations. 

Do  Multiple  Incentives  Really  Work? 

I.  Why  Multiple  Incentives? 

II.  Types  of  Contractual  Incentives 

III.  Multiple  Incentives  Will  Work,  If 

A.  Selection  of  Appropriate  Incentives 

B.  Structuring  of  Incentives 

C.  Proposing  Incentives 

D.  Negotiation  of  Multiple  Incentives 

E.  Multiple  Incentives  as  a Management 
Tool 


Why  Multiple  Incentives? 

The  justification  for  multiple  Incentives  is 
to  focus  management  attention  on  those  costs, 
deliveries,  and  performance  requirements  of 
importance  to  the  Government  by  Incentivizing 
the  contractor  where  it  counts:  Profit. 

Contractors  should  accept  this  challenge 
knowing  that  multiple  Incentives  can  yield 
very  reasonable  profits. 


Types  of  Contractual  Incentives 

Can  be  developed  within  almost  any  contract 
arrangement,  but  usually  developed  within  a 
CPIF  or  FPI  structure. 

Cost  Incentives 

Sharing  Formulas  (Penalty/Reward) 

Targets 

Ceilings 

Floors 

Buffer  Zones 

Schedule  Incentives 

Examples:  End  item  delivery,  test 

completion,  final  acceptance, 
etc. 

Go -No -Go 
Penalty  Only 
Reward  Only 

Performance  Incentives 

Mission  Success-Payload,  Speed,  Range,  etc. 

Independent 

Interdependent 

Other  Incentives 
Design  to  Cost 
Warranties 
Value  Engineering 
Correction  of  Deficiencies 
Termination-Default 
Award  Fees 

Multiple  Incentives  Will  Work  If 

Selection  of  Appropriate  Incentives 
Incentives  Interdependent 

Motivate  contractor  to  strive  for  maximum 
fee  in  all  areas  by  establishing  an 
attainable  range  of  incentive 
effectiveness. 

If  maximum  performance  cannot  be  achieved, 
incentive  structure  should  compel  deci- 
sions be  made  consistent  with  program 
objectives  by  assigning  relative  weights 
to  each  incentive  area. 

Structuring  of  Incentives 

Structuring  of  multiple  Incentive  arrangement 
must  be  done  by  team  effort  (Government  and 
contractor  where  possible). 

Selective  incentive  parameters. 

Identify  minimum  acceptable,  target,  and 
maximum  desirable  level  for  each  parameter. 

Assign  percentage  of  Importance  to  parameters. 

All  incentive  outcomes  should  be  acceptable. 
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Proposing  Incentives 


Firm  definition  and  understanding  of  work 
statement  exists. 

The  solicitation  describes  the  Government's 
minimum  requirements,  performance  goals,  and 
the  type  of  contract  which  is  contemplated. 

Result  in  an  incentive  plan  with  proper 
emphasis  on  incentive  objectives. 

Negotiation  of  Multiple  Incentives 

More  incentive  contract  profit  dollars  can  be 
made  by  a competent  technical-contractual- 
financial  team  throughout  the  incentive  plan 
preparation  and  negotiation  than  by  all  the 
program  management  possible  after  the  contract 
has  been  negotiated. 

The  first  objective  is  to  negotiate  cost 
incentive. 

Negotiate  performance  Incentive. 

• Don’t  forget  exemptions  to  maintain 
' status  quo 

• Causes  beyond  the  control  of  the 
contractor 

e Acts  of  God,  nature,  or  public  enemy 

• Acts  of  omissions  of  other  contractors 
or  the  Government 

• Government  furnished  property 

• Abnormal  economic  fluctuation 

• Unique  circumstances  to  consider  for 
effect  on  incentives 

• Special  funding  requirements 

• Types  of  subcontracts 

• Method  for  effecting  'changes' 

• Method  and  timing  of  payment 

• Negotiate  multiple  incentives  so  that 
maximum  profit  is  attainable  to  achieve 
the  utmost  incentive  goal. 

• Use  multiple  incentives  as  a contract 
management  tool  so  that: 

A.  System  requirements  are  more  clearly 
identified  and  defined. 

B.  Middle  management  and  field  site 
operations  become  more  aware  of  pro- 
gram goals  in  all  areas  of  cost, 
delivery  and  performance. 

C.  Mathematical  models  of  the  incentive 
plan  can  be  formulated  to  permit 
monitoring  performance  goals,  e.g., 
weight,  drag,  and  thrust,  in  relation 
to  technical  progress,  cost  and 
schedule  status. 

Incentives  create  profit  goals  and  there- 
fore are  the  measurements  of  management. 


Multiple  Incentives  as  a Management  Tool 


• Before  incentives,  management  would  not 
want  to  violate  the  contract  but  now 
management  wants  to  fulfill  the  contract. 

• Management  from  top  to  bottom  must 
understand: 

The  goals  or  milestones  that  are 
incentivized 

The  measurement  of  incentive  success 
or  failure 

The  profit  dollar  rewards  associated 
with  success  and  penalties  associated 
with  failure. 
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ORGANIZATIONAL  EFFECT  OF 
ADMINISTERING  INCENTIVES 


Mr.  Hubert  L.  O'Brien 
TRW  Space  Systems  Division 


Mr.  O'Brien  began  by  conceptually  constructing  an 
organization.  The  organization  h.  a purpose;  it 
can  define,  communicate,  and  measure  goals;  and  it 
can  reward  achievement.  He  went  on  to  relate  this 
concept  by  buyers,  sellers,  and  individuals. 

Mr.  O’Brien  gave  several  examples  from  his  corporate 
experience  of  the  various  types  of  incentives  and  of 
some  of  the  problems  encountered.  He  showed  how 
TRW,  in  a couple  of  instances,  through  the  use  of 
multiple  incentives,  was  in  a situation  where  they 
would  have  received  their  greatest  monetary  reward 
for  poor  performance  or  failure. 

He  impressed  on  us  the  requirement  for  continued 
emphasis  on  reasonable  cost  estimating.  He  stated 
that  buyers  and  sellers  perform  for  a variety  of 
reasons  other  than  profit  or  cost  savings. 

A comment  on  profit  or  incentive  sharing  with 
employees  suggested  caution  in  this  area.  Mr. 
O'Brien  stated  that  experience  shows  that  this 
approach  motivates  the  best  individuals  to  gravitate 
toward  the  most  successful  programs  leaving  the 
"problem  program"  in  even  greater  difficulty. 

• Develop  the  concept  of  any  organization  or 
enterprise 

• Organized  for  a purpose 

• Define  goals 

• Communicate  goals 
a Measure  goals 

a Reward  achievement 

a Relate  the  5 concepts  to  the 

a Buying  activity 
a Government  or 
a Industry  prime  contractor 
a Selling  activity 
a People  as  individuals 

a How  contract  incentives  motivate 

organizations  and  people  (with  examples 
and  "war  stories") 

a Cost  incentives 
a Cost  realism 
a Cost  sharing 
a Contract  type  - risk 
a Performance 
a Schedule 
a Award  fees 
a Overlap 

a Mission  success  pools 
a Double  whammies 
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SAMSO  ORGANIZATIONAL  CONSIDERATIONS 
IN  INCENTIVE  CONTRACTING 

Col  John  J.  Caulfield,  Jr. 
Director  of  Procurement  for 
Space  Communications  Systems,  SAMSO 


The  attached  "Mid-Term"  Examination  was  devised  by 
Col  Caulfield  as  a discussion  tool.  The  text  of 
Col  Caulfield's  remarks  are  quoted  in  their  entirety. 

"Since  you  folks  are  approximately  half  way  through 
this  Mission  Assurance  Conference,  I have  facetiously 
organized  my  outline  in  the  form  of  a mid-term 
examination. 


Question  No.  1 


Question  No.  2 

2.  What  is  "...in  the  best  interests  of  the 
Government"? 

A.  Technical  Performance 

B.  Cost 

C.  Schedule 

D.  Socio-Economic  Achievement 

E.  All  of  the  above 

F.  Some  of  the  above 

• By  the  way,  what  is  "...in  the  best 
interests  of  the  Government?" 

• TECHNICAL  PERFORMANCE  is  the  traditional 
answer  - the  school  solution. 
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1.  The  purpose  of  incentive  contracting  is  to 

motivate  the to  performance  which 

is  in  the  best  interests  of  the 


We  begin  with  a "fill-in-the  blanks" 
question. 

I'll  give  you  a hint  - the  two  missing 
words  are  Government  and  Contractor. 

Which  word  goes  where? 

Try  inserting  Government  in  the  first 
blank  and  you  get. . . 

On  the  other  hand,  if  you  insert  the  word 
Contractor  first,  you  get... 

Note  that  grammatically  both  words  work 
equally  well  in  either  position. 

But  SAMSO 's  goal  is  to  motivate  the 
Contractor  to  performance  in  the  best 
interests  of  the  Government. 

Why  then,  does  the  typical  SAMSO  RFP 
contain  a detailed  incentive  plan  which  is 
presented  to  the  contractor  as  a given? 

Perhaps  it  derives  from  an  organizational 
arrogance. 

After  all,  if  the  word  Government  fits  in 
either  of  these  blanks,  then  the  Govern- 
ment should  be  able  to  play  two  roles: 
that  is 

(1)  The  Government  should  be  able  to 

stipulate  what  is  in  the  best  inter- 
ests of  the  Government 


• But  more  and  more  COST  has  become  a major 
determinant  of  program  success. 

• And  in  the  satellite  business,  we  are 
expecting  launch  costs  to  skyrocket  come 
FY81  and  the  Space  Shuttle. 

« Incidentally,  the  Government  *?s  an  array 
of  less  obvious,  more  subtle,  nterests 
which  it  pursues  through  our  systems 
contracts. 

• Hire  the  handicapped 

• Equal  economic  opportunity 

• Small  businesses 

• Minority  enterprises 

• Confidentially,  look  for  future  SAMSO  RFPs 
to  provide  an  award  for  achievement  of 
minority  enterprise  subcontract  goals. 

• I guess  what  I'm  saying  is  that  the  pres- 
sure is  on  our  organization  to  pursue  a 
variety  of  Government  interests. 

Question  No.  3 

3.  How  do  you  measure  technical  performance? 

A.  Meets  or  exceeds  all  specifications. 

B.  Meets  or  exceeds  some  specifications. 

C.  Achieves  design  life  in  orbit. 

D.  Achieves  mean  mission  duration. 

E.  Stores  unobtrusively  in  orbit. 

F.  Stacks  nicely  in  warehouse. 

• Question  No.  3 also  causes  lots  of  organ- 
izational soul-searching. 


and 

(2)  What  will  motivate  the  contractor  to 
achieve  it. 

Maybe  we  ought  to  instead  invite  the  bid- 
ding contractors  to  propose  Incentive 
plans  that  they  find  meaningful. 


e We  paid  the  contractor  to  meet  all  specs., 
why  should  we  now  accept  anything  else? 

• But  all  or  nothing  somehow  doesn't  seem 
fair,  so  why  don't  we  pay  him  for  meeting 
the  spec,  for  critical  modes... or  for 
being  on  the  air  most  of  the  time,  if  not 
all  the  time? 
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By  the  way,  how  long  should  the  incentive 
period  run? 

The  spec,  may  prescribe  a design  life  of 
5 years,  but  the  Failure  Modes  and  Effects 
Analysis  (FMEA)  will  likely  calculate  a 
so-called  MEAN  MISSION  DURATION  of  36 
or  40  months. 

b. 

Organizational ly,  how  do  we  make  the 
critics  of  defense  procurement  understand 
that  we  are  not  giving  away  the  farm  when  c. 
we  agree  to  pay  an  orbital  incentive  at 
the  end  of  3 years,  when  the  design  life 
is  5 years? 

d. 

Sometimes  we  are  faced  with  storing  a 
satellite  in  space,  in  orbit,  in  order  to 
have  it  immediately  available  if  and  when 
needed  in  the  future. 

e. 

How  do  we  handle  that  in  terms  of 
incentives? 

Organizationally,  we  find  it  difficult  to 
justify  paying  an  incentive  for  something  f. 
we  aren't  using. 


a.  Incentives  are  budgeted  for  the  fiscal 
periods  in  which  they  are  expected  to  be 
earned.  Poses  practical  problems. . .Given 
a Federal  Budget  System  that  only  extends 
5 years  into  the  future.  How  do  you 
budget  for  incentives  for  a system  with  a 
10-year  useful  life? 

Advance  payment  of  incentives  is  contrary 
to  public  law. 

Milestone  billing  provisions  for  liquidat- 
ing progress  payments  can  be  a form  of 
incentive. 

The  Design-To-Cost  concept  poses  unique 
incentive  problems.  How  do  you  incenti- 
vize  a production  DTC  Boogy  years  in 
advance  of  signing  a production  contract. 

The  Cost  Plus  Award  Fee  type  contract 
enables  the  Government  to  change  its  mind 
on  its  incentive  priorities  over  the  life 
of  a contract. 

A contractor  gets  discouraged  when  the 
Government  is  unable  to  place  his  satel- 
lites in  orbit. 


Question  No.  4 


How  do  you  reconcile  sometimes  conflicting 
technical  requirements  of  the  various 
payloads  of  multi-mission  spacecraft? 


A.  Carefully 

B.  Other  (specify). 


g.  "Non  Multa  sed  multum. " 

It  is  quality  rather  than  quantity  that 
matters. 

• Lucius  Annaeus  Seneca,  8BC  - 65AD. 


Question  No.  5 


True  or  false: 

A.  Incentives  are  budgeted  for  the  fiscal 
periods  in  which  they  are  expected  to 
be  earned. 

B.  Advance  payment  of  incentives  is  con- 
trary to  public  law. 

C.  Milestone  billing  provisions  for 
liquidating  progress  payments  can  be 
a form  of  incentive. 

D.  The  design-to-cost  concept  poses 
unique  incentive  problems. 

E.  The  cost  plus  award  fee  type  contract 
enables  the  Government  to  change  its 
mind  on  its  incentive  priorities  over 
the  life  of  a contract. 

F.  A contractor  gets  discouraged  when 
the  Government  is  unable  to  place  his 
satellites  in  orbit. 

G.  "Non  Multa  Sed  Multum". 
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ORGANIZATIONAL  EFFECT  OF  ADMINISTERING 
INCENTIVES 

Mr.  George  S.  Pappas 
Director  of  Contracts 
Ball  Aerospace  Systems  Division 


Mr.  Pappas  began  his  remarks  by  stating  that  his 
company's  experience  is  as  a medium-sized  contractor 
with  only  a minor  fraction  of  their  business  in 
Government  Aerospace  contracts. 

He  suggested  that  to  evaluate  the  results  of  con- 
tractual incentives,  one  must  ascertain  what  was 
done  differently  because  of  the  incentives.  Any 
review  of  this  nature  will  get  a distorted  picture 
because  of  the  other  variable  factors  present.  That 
is  to  say,  that  it  is  nearly  impossible  to  isolate 
two  situations  which  are  identical  except  for  the 
incentives  and  to  identify  the  effects  thereof. 

In  Mr.  Pappas'  experience,  he  sees  no  trade-off 
analyses  being  made  of  incentive  factors.  This 
elicited  several  instances  of  contrary  experience 
from  others  in  the  audience. 

Mr.  Pappas  stated  that  profit  sacrifices  are  made  in 
the  process  of  acquiring  and  performing  a contract, 
but  there  are  practical  limits  to  these  sacrifices. 
From  his  perspective,  the  contractor  will  occasion- 
ally take  a low  profit  rate  if  the  contract  will 
allow  the  corporation  to  broaden  their  technical 
expertise.  This,  he  asserts,  is  one  of  the  motiva- 
tions for  Government  business. 

Mr.  Pappas  feels  that  extra-contractual  considera- 
tions such  as:  expansion  of  operations;  reputation 
building;  and  coverage  of  corporate  GSA  expenses; 
dominate  over  profit  and  fee. 

Mr.  Pappas  made  the  following  points  about 
incentives: 


1. 

There  is  no  significant  correlation 
between  cost  sharing  ratios  and  overruns/ 
underruns. 

z. 

Incentives  are  very  costly  to 

administer. 

3. 

Contractors  will  not  sacrifice 
for  profit. 

performance 

4. 

Incentives  do  not  work  to  the 
of  the  Government. 

disadvantage 

5. 

Incentives  serve  as  a planning 
for  DOD  personnel. 

discipline 

6. 

Incentives  clearly  communicate 
objectives  to  the  Contractor. 

Government 

In  his  closing  remarks,  Mr.  Pappas  states  that  "The 
Government  never  aets  what  it  contracts  for  in  CPFF 
contractinq.  It  does  qet  what  it  wants  because 
contracts  are  ill-defined  early  on." 
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INCENTIVE  INNOVATIONS 

Lt  Col  Michael  M.  McMillan 
HQ  AFSC 


Lt  Col  McMillan  began  his  presentation  by  discussing 
current  trends  within  the  AFSC  acquisition  environ- 
ment. Most  of  the  items  covered  are  of  special 
interest  to  the  new  AFSC  Commander,  General  Slay. 

Some  observations  made  by  Lt  Col  McMillan: 

1.  The  Government  does  not  do  a good  job  of 
preparing  and  sharing  "Lessons  Learned" 
in  the  systems  acquisition  process. 

2.  As  presently  used,  the  Business  Strategy 
Panel  is... "an  exchange  of  ignorance" 
because  one  of  the  major  entities  con- 
cerned is  missing;  the  contractor. 

3.  The  key  to  a successful  program  is:  Early 
Identification  of  Program  Objectives. 

4.  There  are  no  innovations.  We  must  learn 
to  apply  what  we  have  more  effectively. 

5.  The  use  of  the  Draft  RFP  is  the  contrac- 
tor's opportunity  to  share  opinions/ 
approaches  with  the  customer. 

6.  We  must  all  (buyer/seller)  do  a better  job 
of  trade-off  analysis. 

7.  The  Government  must  do  a much  better  job 
of  identifying  alj_  contractor  motivations. 

The  AFSC  Acquisition  Environment 

A-109  Implementation 

Firm-up  cost,  schedule,  performance  baselines - 
changes  require  formal  direction/ funding 

Renewed  emphasis  on  QA  in  program  offices 

Past  Performance  - Major  area  in  source 
selection 

Shift  to  FFP  contracts  for  follow-on  production 
Shorten/simplify  procurement  leadtime 
CS2  3rd  level  of  WBS 
What  Really  Motivates  Contractors? 

Profit? 

Cash  flow? 

Pride  in  products  (S ) ? 

Public  opinion? 

Increased  share  of  the  market? 


r 
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"INCENTIVE  CONTRACTING  IN  SPACE" 

Mr.  P.  Nino  Noal 
Director  of  Contracts 
Defense  Systems  Division 
Hughes  Aircraft  Co. 


Mr.  Noal  observed  that  the  eleventh  speaker  in  our 
all-day  workshop  runs  the  risk  of  repeating  much  of 
what  has  been  said.  Mr.  Noal's  presentation  served 
as  a valuable  summary  and  underscored  several  impor- 
tant points. 

Some  comments  of  particular  interest  to  the  audience 
were: 

1.  Good  corporate  publicity  is  a major  moti- 
vator for  contractors. 

2.  A concept  worth  pursuing  is  the  total 
application  of  an  Award  Fee  to  Orbital 
Performance. 

3.  A lot  of  floor  discussion  was  generated 
when  Mr.  Noal  suggested  that  some  circum- 
stances may  warrant  use  of  the  "Umbrella" 
concept  for  major  subcontractors.  In  this 
situation,  all  major  subcontractors  are 
incentivized  in  equal  proportion  to  the 
prime. 

4.  Mr.  Noal  briefly  mentioned  the  use  of  the 
"Martin  Incentive"  discussed  earlier  by 
Mr.  Crotser.  He  confirmed  that  this  has 
been  shown  to  be  effective  when  properly 
administered. 

5.  Since  space  missions  now  normally  last 
from  three  to  seven  years,  how  do  we  shift 
incentive  award  criteria  to  match  mission 
objectives  which  change  during  this 
period? 

6.  Probably  the  best  incentive  structure  is 
one  which  incorporates  a proper  mix  of 
multiple  incentives  and  an  award  fee. 

Incentive  Contracting  in  Space 

• Contractor  must  have  stake  in  mission 

success 

• Incentive  must  be  heavily  slanted  on 
performance 

• Continuing  emphasis  on  innovative  ideas 
important 

• Susceptible  to  stereotyping 

All  Award  Fee  on  Orbital  Performance 

• Leaves  most  tradeoffs  to  contractor 

• Must  have  "structure"  in  award  fee  plan  at 
start  of  program 

• Contractor  must  "feel"  program  objectives 


• Requires  thoughtfulness  and  program 
integrity 
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"Umbrella"  Concept  for  Major  Subcontracts 

• Where  subcontractor  effort  is  critical  to 
success 

• Creates  closer  identification  with  mission  . ij 

success 

• Forces  continuing  cooperation 

"Making  Out  A Check" 

• Does  it  get  management  attention? 

• Can  it  cause  ruin  through  inattention? 

Matching  Changing  Mission  Objectives 

• Incentive  criteria  fixed  3-7  years  prior 
to  mission  performance  period 

• Mission  objectives  can  change  or  shift 
emphasis 

• Incentives  can  become  "fee-measures"  only 

• Award  fee  or  other  ideas  to  treat  the 
problem 

Multiple  Incentives  Plus  Award  Fee 

• Not  really  new  - one  early  example  was 
NASA's  Surveyor  program 

• Allows  continuing  government  influence 
throughout  the  program  - changing  emphasis 
as  appropriate 

• Difficult  to  structure  - tradeoffs  are 
agonizing  - complex  to  administer  - but 
probably  the  best 

COtRCRCIAl  CONCEPTS  ARMED  TO  MILITARY  PROGRAMS 


• TENDS  TO  SHIFT  MISSION  ASSURANCE  RESPONSIDILITY  TO  CONTRACTOR 

• EX/WLE: 


WORKSHOP  SUMMARY 
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WORKSHOP  SUMMARY 


Summary  Remarks 

The  following  observations  and  recommendations  were 
made  as  a result  of  the  Contract  Incentive  Workshop: 

• All  parties  involved  in  creating  and 
administering  multiple  incentives  must  be 
cautious  not  to  create  a counter- 
producti ve  incentive  structure.  We  must 
insure  that  tradeoff  analyses  do  not 
result  in  motivation  which  tend  to  degrade 
the  mission.  Action:  SAMSO/ Industry 

• To  determine  whether  contractual  incen- 
tives truly  motivate  a contractor,  many 
factors  must  be  evaluated.  It  can  be 
said,  however,  that  they  will  be  ineffec- 
tive if  they  are  too  complex  and  not  fully 
understood  by  both  parties.  A straw  vote 
within  the  workshop  on  the  question,  "Do 
Contract  Incentives  Motivate?"  resulted  in 
a nearly  equal  split  between  yes,  no, 
maybe.  Action:  SAMSO;  must  insure  simple, 
easily  understood  incentive  arrangements. 


• There  was  virtually  unanimous  agreement  on 
the  desirability  and  usefulness  of  the 
Award  Fee.  It  was  felt  that  the  use  of 
periodic  awards  provides  real  time  feed- 
back to  the  contractor  on  his  performance. 
It  also  allows  the  flexibility  to  adjust 
award  criteria  with  changing  government 
requirements  and  emphasis  over  the  long 
life  of  present  day  space  programs.  Con- 
tractors report  that  top  management 
attention  is  easily  focused  by  the  Award 
Fee  "report  card".  It  was  also  noted  that 
in  the  proper  circumstances,  even  bench 
level  workers  can  share  the  award  and  be 
effectively  motivated.  Recommendation: 
SAMSO  and  Industry  continue  to  use  in  a 
judicious  manner. 

• Another  recurring  theme  heard  throughout 
the  day  was  that  there  are  many,  fre- 
quently more  effective,  incentives  to  con- 
tractors than  the  promise  of  increased 
profit.  Pride  and  publicity  were  men- 
tioned most  frequently  as  top  management 
attention-getters.  It  was  noted  that  a 
successful  space  mission  filters  through- 
out the  corporate  structure.  The  govern- 
ment could  get  a lot  of  mileage  out  of  a 
well  handled  mission  publicity  program 
that  publicly  recognizes  the  contractor's 
contribution.  It  was  also  stated  by 
several  attendees  that  personal  and  cor- 
porate pride  will  always  prevent  a con- 
tractor from  making  profit  enhancing/ 
mission  degrading  decisions.  Other  extra- 
contractual  incentives  mentioned  are: 
positive  cash  flow  (milestone  billings); 
corporate  growth;  increased  market  share; 
follow-on  business;  utilization  of  avail- 
able skills  and  plant  capacity;  and 
absorption  of  fixed  costs. 


• Increased  mission  duration  of  today's 
space  programs  have  resulted  in  incentive 
payments  over  an  extended  period.  Current 
interest  rates  and  inflation  tend  to  dis- 
count the  value  of  the  downstream  incen- 
tive payments.  It  was  felt  that  the 
customer  must  study  alternatives  which 
address  the  time  value  of  money  when 
constructing  long  term  incentive  plans. 
Action:  SAMSO/PM. 

• A recommended  improvement  to  incentive 
plan  preparation  was  discussed.  This 
approach  would  be  to  have  the  customer 
provide  broad  guidelines  and  priorities  to 
be  incorporated  within  the  plan.  With 
this  information  incorporated  in  the  RFP, 
the  contractor  would  then  propose  a 
detailed  incentive  plan.  The  obvious 
advantage  of  this  approach  is  that  the 
contractor  is  the  one  who  can  best  deter- 
mine what  his  major  motivators  are  and 
the  negotiation  process  should  distill 
the  optimum  structure  for  the  particular 
situation.  Action:  Individual  Buying 
Agency.  P.S.  SAMSO/PME,  in  response  to 
this  recommendation,  has  recently  released 
an  RFP  with  this  approach. 

• The  Space  Shuttle  era  will  require  new 
incentive  approaches.  Recoverable  pay- 
loads;  on-orbit  testing;  reentry  surviv- 
ability are  but  a few  of  the  anticipated 
issues  which  will  undoubtedly  affect  the 
present  risk  environment  of  space  pro- 
grams. Recommendation:  Government  and 
Industry  must  jointly  explore  the  Shuttle 
environment  and  recommend  alternative 
incentive  approaches. 
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WORKSHOP  C 

THE  ROLE  OF  PROGRAM  REVIEWS,  INDEPENDENT  REVIEWS 
AND  AUDITS  IN  MISSION  ASSURANCE 

Co-Chairmen  Workshop  Coordinators 

Col.  James  A.  Smith,  SAMSO  Mr.  Edward  Newman,  SAMSO/AW 

Special  Assistant  to  Vice  Commander  Mr.  Dan  Browne,  SAMSO/AW 

Mr.  Sheldon  C.  Shallon 
Chief  Scientist 
Defense  Systems  Division 
Hughes  Aircraft  Company 

AGENDA 


Wednesday, 

0830-0845 

0645-0910 

0910-0930 

0930-0950 

0950-1010 

1010-1030 

1030-1050 

1060-1130 

1130-1300 

1300-1700 

1700 


April  26 

Workshop  Overview 

The  SAMSO  Program  Manage- 
ment Review  Process 

Industry  Corporate  Visibility 
for  Mission  Success 

Progressive  Acceptance  Reviews 
and  Audits 

Break 

Role  of  Continuous  Internal 
Audits 

Internal  Review  - A Management 
Control 

Contractor's  Approach  to 
Independence  in  Program 
Reviews 

Lunch 

Summary  of  Workshop  Objectives 
and  Open  Discussion 

Adjourn 


Col.  James  A.  Smith,  SAMSO 

Lt.  Col.  Gordon  L.  Carpenter,  SAMSO 

Mr.  Mark  Sasso,  RCA 

Mr.  Frank  J.  Wolf,  Hughes 

Mr.  Heinie  Shaw,  TRW 

Mr.  Walter  L.  Finch,  Lockheed,  LMSC 

Mr.  Ben  J.  Wier,  General  Dynamics 
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THE  ROLE  OF  MANAGEMENT  AND 
THE  PROGRAM  REVIEW  PROCESS  IN 
MISSION  ASSURANCE 

Col.  James  A.  Smith,  SAMSO 
Special  Assistant  to  Vice  Commander 


Objectives: 

1.  Exchange  views  within  industry  on  corpor- 
ate program  review  policies,  techniques, 
and  practices  which  appear  to  enhance 
mission  assurance. 

2.  Assess  if  present  industry/government 
review  process  is  optimum  for  mission 
assurance  - recommended  changes. 

Discussion  Areas: 

1.  Involvement  of  senior  staff  and  line 
management  to  what  degree  of  program 
detail . 

2.  Timing  of  senior  management  involvement 
(periodic,  key  milestones,  crisis 
oriented) . 

3.  Industry  review  teams'  contribution  to 
mission  assurance,  timing,  and  need  for 
independence. 

4.  Involvement  of  prime  management  in  review 
process  of  subcontractors. 

5.  Need  and  role  for  special  reviews  and 
audits. 

6.  Usefulness  and  timing  of  government 
review  process  in  mission  assurance. 

7.  Roles  and  mix  of  industry/government 
review  teams  - keep  separate  or  join 
forces. 

SAMSO  Independent  Readiness  Reviews 

• Initiated  because  of  failures 

• A last  check  on  programs  which  are 

• New 

• Significantly  changed 


• Unusually  complex 
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PROGRAM  REVIEW  BOARD 
MMR 
DMSP 


! 


set. 


Background 

• Number  of  failures/anomalies  occurred  in 
1971 

• SAMSO  Commander  established  a failure 
review  committee 

• Committee  chaired  by  Vice  Commander 

• Objectives  committee 

• Evaluate  all  failures/anomalies  for 
minimum  of  5 years 

• Review  current  development  programs 

• Development  working  groups 

• Make  recommendations 


SAMSO  MANAGEMENT  REVIEW  PROCESS 

Lt.  Col.  Gordon  L.  Carpenter, 
SAMSO,  Chief,  Problem  Analysis  Center 


Management  Actions 

• 1972  failure  review  determined 

• Lack  of  management  involvement 

i Poor  quality  programs 

• Design  and  test  deficiencies 

t Poor  quality  parts  used 

• Four  working  groups  organized 

• Management  working  group 

• Quality  working  group 

• Design/test  and  human  procedural  work- 
ing group 

• Piece  parts  working  group 

Management  Working  Group 

• Problems  found 

• Lack  of  incorporation  of  lessons 
learned  from  other  programs. 

• Need  for  more  management  involvement 
throughout  life  of  program. 

• Infuse  corporate  experience  into  each 
program. 

• Recommended 

• Development  of  commanders  policy 
statements  - SAMSOR  5-4 

• Development  of  a corporate  review 
process  - SAMSOR  800-3 

• Reorganize  directorate  of  acquisition 
support. 


Status  of  Commander’s  Policies 

FLIGHT  FAILURES  / ANOMALIES  BY  CATEGORY  SAMSOR  5-4 


Date  Signed 

1. 

Independent  Stability  and 
Control  Analyses 

27  Mar  75 

2. 

Participation  in  Program 
Planning,  Acquisition  and 
Test 

18  Sep  72 

3. 

Subcontractor  Surveillance 

17  Oct  75 

4. 

Change  to  Proven  Launch 
Vehicle  Hardware  and 
Software 

29  Sep  75 

5. 

Control  of  Launch  Vehicle 
Inventory 

29  Sep  75 

1 
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Date  Signed 


6. 

Quality  Assurance 

27  Mar  75 

7. 

Microelectronics  for 

Space  and  Missile  Systems 

18  Aug  75 

8. 

Independent  Structural 

Load  Analysis 

27  Mar  75 

9. 

Management  of  Computer 
Resources 

5 Jan  77 

10. 

Accident  Risk  Management 

19  Sep  77 

11. 

Management  Review  System 
for  Small  Contracts 

28  Feb  75 

12. 

Space  Vehicle  Testing 

3 Mar  75 

13. 

Independent  Readiness 

Review 

4 Jan  78 

14. 

Independent  Stress  and 
Fracture  Analysis 

24  Jul  75 

15. 

Production/Manufacturing 

17  Oct  75 

16. 

Pre-Award  Surveys 

17  Nov  75 

17. 

Business  Strategy  Panels 

24  Jun  76 

18. 

System  Reviews  and  Audits 

2 Nov  77 

19. 

Failure  Modes  and  Effects 
Analysis 

27  Dec  77 

Management  Review 
(SAMSOR  800 3) 

• Establish  program/planning  review  board 

to  accomplish: 

• Planning  reviews 

• Review  SAMSO  mission  areas  to 
determine  what  programs  should  be 
developed 

• Precontract  reviews 

• Review  each  major  acquisition 
before  award  of  contract 

• Design  certification  reviews 

t Review  each  major  acquisition  after 
critical  design  review 

• Mission  readiness  reviews 

• Review  each  major  acquisition  pro- 
gram after  acceptance  but  prior  to 
launch 

• Launch  readiness  reviews 

• Program  manager  and  launch  agency 
jointly  review  each  major  acquisi- 
tion program  prior  to  launch 

• Indpeendent  readiness  reviews 

• Selectively  review  program  when 
directed  by  commander 


REVIEW  SEQUENCE 
ON  A 

TYPICAL  ACQUISITION  PROGRAM 


iTlMf  tvA»MOMt*ftNlACHfVCNt  itMOCftAMMFfftOCNTi 
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Summary 


Quality  Working  Group 
Problems  Found 

• Weak  quality  assurance  programs 

• Not  enough  definitive  QA  contract 
requirements 

• Lack  of  quality  surveillance  by  govern- 
ment and  contractors 

Recommended 

• Augmentation  of  MIl-Q-9858  with  space 
hardware  special  provisions 

• Involve  CAQ/QA  personnel  in  design  and 
engineering  test 

• Upgrade  quality  of  DCAS/AFCMD/PO  quality 
assurance  personnel 

• Establish  program  office  quality  assur- 
ance manager 

• Strengthen  subcontractor  management 
Design/Test  and  Human  Procedural  Working  Group 
Problem  Found 

• Design  not  fully  checked 

• Testing  Requirements  not  consistent 

• Procedures  not  checked  and  followed 

Recommended 

• More  detailed  emphasis  on  PDR  and  CDR 

• Establish  a common  space  vehicle  testing 
std  (MIL- STD- 1540) 

• Strengthen  review  of  procedures  both  in 

plant  and  at  launch  site 

Piece  Parts  Working  Group 

• Problems  found 

• Contractors  not  selecting  good  quality 
parts  in  design  phase 

i Contractors  not  managing  selection  of 
parts 

• Quality  parts  not  available 

• Recommended 

• Make  quality  parts  available  for  use 
by  contractor 

• Insure  management  of  parts  selection 
accomplished  by  contractor 

• Develop  special  design  and  test 
requirements  for  space  parts 

• Develop  a SAMSO  qualified  parts 
selection  list 

Implementation 

• All  recommendations  made  by  working 

groups  were  approved 

• In  1975,  SAMSO  Vice  Commander,  directed 
that  independent  readiness  reviews  be 
conducted 

• On  programs  as  appropriate 

• About  same  time  as  mission  readiness 
review 


Management  Reviews 

• Improve  mission  success  potential 

• Provide  management  help  if  needed 

• Insure  better  organization  interfaces 

• Provide  visibility  on  lessons  learned 

• Insure  AF  objectives  satisfied 
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MANAGEMENT  RESPONSIBILITIES  AND  PARTICIPATION 

Mr.  Mark  Sasso, 

Manager, 

Satellite  Programs  RCA,  Astro  Electronics 


The  objective  of  this  discussion  is  to  identify 
appropriate  management  involvement  in  the  program 
review  process  in  order  to  provide  timely  utiliza- 
tion of  corporate  resources  in  guaranteeing  mission 
assurance. 

The  appropriate  timing  of  program  reviews  will  be 
identified  during  the  life  cycle  of  a program  to 
provide  management  emphasis  during  critical  design, 
manufacture,  and  test  phases. 

I will  also  discuss  the  involvement  of  functional 
management  disciplines  in  the  review  process  to 
assure  application  of  appropriate  expertise  in  the 
identification  and  solution  of  all  problem  areas. 

In  order  to  provide  a perspective  for  this  discus- 
sion allow  me  to  say  a few  words  about  the  RCA 
Government  Systems  Division  and  how  we  are 
currently  organized. 

Astro-electronics  which  is  part  of  the  Government 
Systems  Division  has  just  celebrated  its  20th  year 
in  space.  Shortly  after  Sputnik  ushered  in  the 
space  age,  RCA  drew  upon  its  broad  spectrum  of 
scientific  and  technological  skills  and  resources 
to  establish  the  Astro-Electronic  Products  Division 
in  March  1958.  In  doing  so,  RCA  became  the  first 
in  the  electronics  industry  to  commit  itself  to  a 
major  role  in  the  new  space  frontier.  Astro- 
Electronics  was  charged  with  the  responsibility 
"to  develop  and  produce  earth  satellites,  space 
vehicles,  and  their  associated  ground  equipment." 

Actually,  RCA's  experience  in  satellite  and  space 
research  dates  back  to  the  early  1950' s when  it 
performed  various  feasibility  studies  for  the  U.S. 
Government.  These  studies  dealt  with  subjects  such 
as  earth-space  exploration,  meteorology,  naviga- 
tion, and  early  warning  systems. 

During  its  20  years  of  space  leadership,  RCA  Astro- 
Electronics  contracted  to  design  and  build  116 
spacecraft,  plus  having  the  responsibility  for  the 
development  of  38  major  systems. 

Over  this  same  span,  RCA  has  successfully  orbited 
72  satellites  and  has  provided  330,640  manhours  of 
post  launch  on-orbit  operational  support.  The 
RCA-built  satellites  have  accumulated  102  years  of 
orbital  operational  life. 

In  addition  to  the  four  product  units  of  which 
Astro-Electronics  is  one,  there  are  five  main  staff 
functions  which  provide  special  resources  which 
can  be  drawn  upon  as  required  by  the  product  units. 
I will  show  later  the  significance  of  this  in  the 
review  process. 

I particularly  want  to  mention  the  staff  engineer- 
ing function  which  in  addition  to  the  advanced 
technology  lab  contains  specialists  in  microtech- 
nology, parts,  materials,  and  reliability 
engineering. 

Astro-Electronics  is  organized  as  a matrix  of 
functional  and  product  operations.  Each  project  is 
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organized  under  a product  business  unit.  Projects 
are  essentially  line  organizations  with  all  the 
necessary  technical  skills  to  satisfy  the  program 
requirement.  They  are  supported,  however,  by  the 
functional  operations  in  technical  areas  requiring 
highly  specialized  skills  such  as  structure 
analysis,  materials,  parts  usage,  and  product 
assurance. 

At  RCA  all  programs  receive  extensive  management 
attention  at  all  levels.  The  philosophy  is  to 
conduct  ongoing  reviews  in  the  primary  interest  of 
avoiding  problems,  not  relaying  exclusively  on  the 
program  manager  to  advise  of  problems  after  they 
occur. 


Hence,  during  the  life  cycle  of  a program,  monthly 
reviews  are  conducted  by  the  program  manager  for 
the  group  Vice  President  and  his  staff  (one  level 
above  the  division  Vice  President).  Of  primary 
concern  to  him  are  technical  problems  causing  or 
about  to  cause  program  impact  and  to  assure  appro- 
priate technical  resources  are  being  applied.  He 
is  also  interested  in  subcontractor  problems  and 
whether  appropriate  levels  of  management  are 
involved  in  problem  solutions.  Subcontractor 
effort  is  also  reviewed  in  much  the  same  manner  as 


our  own.  During  PDR  and  CDR  staff  reliability 
engineering  and  special  technical  support  is  pro- 
vided to  augment  our  own  technical  staff.  In 
addition  we  retain  a consulting  staff  in  special 
areas  of  expertise  to  gain  adde^  assurance  of 
technical  integrity.  He  is  also  concerned  about 
current  or  potential  customer  concerns  and  whether 
there  is  an  appropriate  corporate  response.  When 
problems  arise  that  cause  serious  impact  in  meeting 
system  objectives  he  will  command  appropriate  appli- 
cation of  resources  and  status  reviews  will  become 
more  frequent. 


The  Division  General  Manager  al 
reviews  with  his  staff.  He  is 
the  programs  ability  to  meet  al 
ments,  but  he  will  command  all 
resources  to  assure  timely  and 
solutions.  He  will  also  sol i ci 
when  the  situation  demands.  Hi 
timely  audits  of  all  functional 
ing,  manufacturing  and  product 
problems. 


so  conducts  detail 
also  interested  in 
1 mission  require- 
necessary  divisional 
proper  technical 
t corporate  resources 
s staff  will  make 
processes,  engineer- 
assurance  to  avoid 


At  the  operational  level  the  business  unit  manager 
conducts  weekly  reivews  of  program  status.  To  pro- 
vide a fine  grained  analysis  of  program  health  and 
to  prevent  small  problems  from  becoming  major  dis- 
asters. This  provides  a continuing  look  at  detail 
which  is  absolutely  essential  in  order  to  guarantee 
program  and  mission  success. 

The  program  manager  never  sleeps.  His  is  a daily 
surveillance  of  detail  program  functions.  No 
detail  is  too  small,  and  on  his  shoulders  falls  the 
ultimate  responsibility  for  providing  the  visibility 
of  and  response  to  program  needs.  His  is  a lonely 
job,  but  he  has  all  the  clout  necessary  to  bring 
appropriate  management  attention  to  bear  in  pro- 
viding system  operational  readiness. 

In  addition  to  the  regularly  scheduled  reviews,  we 
have  an  ongoing  quality  and  safety  audit  program 
conducted  by  our  product  assurance  function.  These 
are  scheduled  at  irregular  intervals  and  provide 
assurance  that  appropriate  processes  and  standards 
are  adhered  to  in  the  fabrication  and  test  of  the 
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product.  Here  again  special  technical  support  is 
provided  by  the  staff  functions. 

The  timing  and  frequency  of  the  reviews  as  I have 
just  described  would  appear  to  be  extremely  burden- 
some from  a program  management  point  of  view. 

Indeed  very  often  it  seems  as  though  an  excessive 
amount  of  effort  is  spent  in  preparing  and  provid- 
ing review  data.  Program  management  however,  as 
we  all  know,  is  a constant  surveillance  process  and 
while  we  may  not  always  succeed,  we  do  manage  to 
choose  the  timing  for  the  reviews  so  as  to  minimize 
duplicate  effort. 

I must  say  a few  words  on  the  use  of  and  the  value 
of  independency  in  industrial  reviews.  First,  the 
control  function  established  through  quality  con- 
trol and  safety  reviews  are  tools  for  independent 
reviews  for  our  management.  Also,  other  staff 
functions  are  organizationally  established  for 
independent  reviews,  such  as  the  engineering  staff 
reporting  to  the  general  management.  They  partici- 
pate and  attend  design  reviews  and  are  available 
to  perform  independent  analysis  of  designs  and 
spacecraft  anomalies  for  both  the  program  office 
and  line  management.  Another  independent  reviewer 
is  the  customer  who's  program  office  personnel  and 
his  supporting  technical  and  operational  support 
advisors  review  the  program  in  considerable  detail. 
A separate  but  formal  independent  review  team, 
formed  by  the  customer,  normally  covers  the  same 
ground  twice  but  like  any  reviews  makes  contribu- 
tion. For  complex  and  costly  space  system  now 
being  produced,  all  reviews  contribute  to  the 
success  of  a program.  The  problem  for  industrial 
management  is  to  support  the  review  while  minimiz- 
ing the  impact  on  cost/schedules  and  mission  needs. 

We  at  RCA  welcome  the  design  reviews.  The  mission 
readiness  and  launch  readiness  reviews  are  also 
productive.  Our  concern  is  with  improper  timing 
where  resources  are  strained  at  the  expense  of 
hardware  readiness.  Our  recommendations  are  to 
plan  each  review  early  and  to  cover  it  in  initial 
contract  negotiation  so  that  adequate  preparation 
can  be  made. 

May  I say  in  closing  that  industry's  role  in 
mission  assurance  is  to  be  the  customer  conscience 
to  assure  design  and  hardware  integrity.  To 
assure  through  application  of  design  and  control 
procedures,  design  reviews  and  audit  programs  that 
mission  requirements  are  met. 

To  maintain  data  systems  that  provide  history  of 
test  and  material  discrepancies  and  demonstrate 
proper  closeout  action.  To  assure  the  conduct  of 
detailed  test  data  analysis  that  provides  proof  of 
performance  compliance.  To  provide  continuing 
trend  analysis  leading  to  appropriate  action  in  the 
elimination  of  incipient  failures.  To  maintain 
effective  change  control  systems  guaranteeing 
control  of  system  configuration. 

Finally  to  present  to  the  customer  a system  totally 
defined  as  to  its  health  and  configuration  by  a 
scrupulous  review  process  leading  to  his  confidence 
in  meeting  mission  objectives. 

Thank  you. 
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PROGRESSIVE  ACCEPTANCE 

Frank  J.  Wolf 
Program  Manager 
Hughes  Aircraft  Company 


Flight  Hardware  Cycle 

A simplified  flight  hardware  flow  diagram  is 
shown  which  progresses  from  unit  assembly  and  test, 
through  subsystem  assembly  and  test,  to  system 
assembly  and  test,  and  eventually  to  launch  opera- 
tions and,  hopefully,  results  in  mission  success. 

This  2 to  3 year  activity  is  supported  or,  as  some 
skeptics  would  say,  crutched  by  a series  of  planned 
and  unplanned  reviews  or  audits. 

Not  atypical  of  Parkinson's  law,  the  number  and 
type  of  planned  reviws  are  functions  increasing 
with  time.  Old  reviews,  like  old  soldiers,  never 
go  away,  but  new  reviews  are  added  contractually. 
And  this  assures  failure-free  orbital  operations. 


Let  me  now  address  our  concept  of  progressive 
acceptance  as  one  technique  for  more  effective 
reviews.  Also,  I want  to  point  out  that  one  area 
of  testing  has  generally  been  overlooked.  At  the 
risk  of  being  guilty  of  recommending  another  set  of 
reviews,  let  me  suggest  it  as  a means  of  increasing 
test  effectiveness. 


Progressive  Acceptance 

What  is  progressive  acceptance?  It  is  really 
nothing  more  than  a series  of  adequately  planned 
and  properly  executed  joint  Government-Contractor 
review  boards  operating  throughout  the  program  life- 
time, forcing  decisions  at  each  meeting  and  sifting 
significant  data  so  that  acceptable  data  is 
reviewed  in  a timely  fashion  but  not  carried  for- 
ward, and  so  that  liens  or  postponed  actions  are 
highlighted  and  carried  forward  to  the  next  review 
gate. 


Right? 

Wrong! 


Typical  Orbital  History 

A 2 year  old  study  based  on  21  Hughes  satellites 
whose  comulative  orbital  life  exceeded  81  orbital 
years  showed  that  64  percent  of  orbital  failures 
could  have  been  prevented,  e.g.,  those  associated 
with  design  and  manufacturing  problems. 

Because  of  redundancy,  conservation  designs,  and 
conscious  overdesign,  success  in  terms  of  either 
incentive  earnings  or  fulfillment  of  the  required 
missions  was  well  over  90  percent.  Nevertheless,  a 
large  fraction  of  on-orbit  failures  escaped  the 
scrutiny  of  reviews,  audits  and  tests.  This  is 
prima  facie  evidence  that  we  need  more  reviews  and 
more  tests.  Let  me  suggest  that,  instead,  we  need 
more  effective  reviews  and  more  effective  tests. 


Effective  Reviews 

The  general  thesis  is  that  effecitve  reviews  are 
timely,  well  scoped,  and  staffed  with  good  people 
The  specific  thesis  is  how  we  have  applied  these 
principles  to  the  progressive  acceptance  concept. 
In  the  broader  sense,  maybe  you  can  apply  these 
precepts  to  other  types  of  reviews  or  audits 
inherent  in  a space  vehicle's  life  cycle. 
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Timel iness 


T 


All  progressive  acceptance  reviews  are  preplanned 
and  scheduled  well  in  advance  to  occur  at  the 
following  times: 

• Unit/subsystem  - Approximately  2 to  3 

weeks  after  completion 
of  unit/subsystem  flight 
acceptance  test. 

• System  level  - The  first  system  level 

review  occurs  about  2 to 
3 weeks  before  system 
level  thermal -vacuum 
testing. 

- The  second  system  level 
review  is  held  about  2 to 
3 weeks  prior  to  shipment 
to  the  launch  site  or 
prior  to  spacecraft 
storage. 

- Each  of  these  system  level 
reviews  is  followed  by  one 
or  more  mini-reviews  to 
track  data  to  the  critical 
event. 


• System  level  - Descoped  to  that  series  of 
events  and  tests  which 
have  occurred  since  the 
last  review,  plus  those 
items  intentionally 
carried  forward  from  the 
last  review. 


NARROW  AS  MUCH  AS  POSSIBLE 

• APPLY  SALEABLE  ITEM  CONCEPT 

• SHORTEN  TIME  PERIOD  COVERED 

• REVIEW  ONLY  NEW  DATA  PLUS 
THAT  WHICH  PASSED  THROUGH 
LAST  GATE 


• PLANNED 

• SCHEDULED  WELL 
IN  ADVANCE 

• CLOSE  TO  EVENT 


Scope 

The  scope  of  each  review  is  narrowed  as  much  as 
possible.  Within  reasonable  limits,  a few  more 
reviews,  each  with  limited  scope,  are  more  effec- 
tive than  one  review  covering  the  entire  produc- 
tion cycle.  Thus,  progressive  acceptance  reviews 
are  limited  in  scope  as  follows: 

• Unit/ subsystem  - Descoped  to  that  box  or 
subsystem  which  repre- 
sents a saleable  item(s) 
from  the  functional  area 
to  the  system  area. 

Since  the  saleable  item 
concept  is  applied,  the 
review  covers  the  single 
item  or  group  of  items 
tested  by  the  functional 
area  prior  to  delivery 
to  the  system  area. 


Source  Data 


Since  each  of  the  progressive  acceptance  reviews  is 
limited  in  scope,  there  is  opportunity  for  cover- 
ing that  scope  in  depth.  Therefore,  the  a priori 
provided  source  data  is  extensive.  It  islilso 
feasible  to  extend  the  source  data  should  the 
board  so  decide.  On  a first  article  production 
item,  it  would  be  normal  to  spot  check  or  sample 
some  aspects  of  PDR,  CDR,  or  the  qualification 
hardware.  On  other  occasions,  it  is  in  order  to 
spot  check  closed  trouble  and  failure  reports  co r 
adequacy  of  corrective  actions. 

t Test  data,  including  deviations  or 
waivers 

• Open  trouble  and  failure  reports 

• All  prior  trouble  and  failure  reports 
closed  "cause  unknown" 

• Open  action  items  from  previous  accep- 
tance reviews 

• Closed  action  items  from  prior  review 

• Authorized  but  unincorporated  and 
potential  engineering  changes 

• Configuration  changes  from  last  produc- 
tion unit 

• Open  vehicle  discrepancy  log  items 
(system  level  only) 
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Personnel 

This  most  critical  element  can  be  analyzed  in  four 
subtopics: 

1.  Get  the  best  people  you  can,  but  limit 
the  number  to  5 or  7. 

2.  In  striving  for  the  best  people  available, 
consider  using  some  independent  board 
members  to  obtain  objectivity  and  cross- 
fertilization. 

Exclusive  user  of  independent  members, 
however,  is  cumbersome  and  time-consuming. 
It  is  desirable  that  independent  members 
be  available  for  future  reviews  to  assure 
continuity  and  minimize  indoctrination. 

3.  It  is  also  advantageous  to  have  continuous 
SPO  and  Aerospace  involvement.  The 
Customer  does  not  like  surprises,  and  his 
continuous  sharing  of  Contractor  pro- 
blems has  a net  positive  effect.  Don't 
let  the  "dirty  linen"  syndrome  sway  you 

to  exclude  the  SPO  at  each  step  of  the 
progressive  acceptance  path. 

4.  Being  prepared  for  the  review  is  another 
key  ingredient  for  achieving  effective- 
ness. The  board  chairman  must  insist 
upon  mandatory  review  of  the  data  pack- 
age before  the  formal  review.  If  a pre- 
meeting session  is  in  order,  the  chairman 
should  call  it.  And  most  important  is 
insistence  upon  written  action  items  or 
questions  prior  to  the  meeting. 


• THE  BEST 


• INDEPENDENCE  IS 
NOT  PANACEA 


• CUSTOMER 
INVOLVEMENT 


• ADEQUATE 
PREPARATION 


Board  Composition 

The  chairmanship  and  composition  of  the  boards  are 
based  on  the  buyer-seller  principle.  Typically, 
these  are: 

• Unit/Subsystem  Acceptance 

• Chaired  by  Chief  Program  System 
Engineer 

• Data  presented  by  each  responsible 
engineering  activity 

• Review  Board  consisting  of  technical 
experts  from  Contractor,  SPO,  and 
SPO  technical  consultants  related  to 
technical  subject  under  consideration. 

• Consent- to-Thermal -Vacuum  Test 

• Chaired  by  Vehicle  Manager 


• Data  presented  by  Chief  Program  System 
Engineer 

• Review  Board  consisting  of  technical 
experts  from  Contractor,  SPO,  and  SPO 
technical  consultants 

• Consent-to-Ship 

• Co-chaired  by  Contractor  Program 
Manager  and  SPO  Program  Manager 

• Data  presented  by  Vehicle  Manager 

• Review  Board  consisting  of  senior 
technical  experts  from  Contractor, 

SPO,  and  SPO  technical  consultants 


• BUYER  SELLER  PRINCIPLE 

• JOINT  CUSTOMER  CONTRACTOR 

• TECHNICAL 

• SOME  INDEPENDENT 
EXPERTS.  IF 
APPROPRIATE 


Documentation 

While  documentation  is  time-consuming  and  expensive, 
it  nevertheless  is  mandatory. 

A data  package  prepared  from  the  source  documents 
is  required  so  that  board  members  can  adequately 
prepare  themselves.  Written  minutes  are  needed  to 
document  the  rationale  for  decisions  reached. 

Written  action  item  assignment  and  written  action 
item  closure  are  necessary  for  clarity. 

Careful  attention  to  preparation  of  these  docu- 
ments is  invaluable  for  future  reference  and  pre- 
cludes duplication  of  effort. 

• Data  package 

• Minutes  of  meetings 

• Written  action  item  assignment 

• Written  action  item  closure 


Applicability  of  Progressive  Acceptance 

The  concept  of  progressive  acceptance  is  applicable 
over  a wide  range  of  organizational  relationships 
or  types  of  end  items.  At  Hughes,  we  have  used 
progressive  acceptance  to  incrementally  accept 
major  subcontracted  subsystems.  Even  in  a first- 
tier  subcontract,  we  find  it  desirable  to  have  SPO 
participation.  Progressive  acceptance  is  used 
routinely  for  in-house  hardware  builds.  More  radi- 
cal may  be  the  concept  of  treating  data  processing 
end  items  like  space  hardware  end  items.  We  have 
tailored  and  successfully  implemented  progressive 
acceptance  reviews  for  major  data  processing 
developments.  The  concept  is  applicable  to  any 
saleable  end  item. 
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ORGANIZATIONAL 

• PRIME/SUBCONTRACTOR 

• PRIME/IN  HOUSE 

END  ITEMS 

• SPACE  VEHICLES 

• DATA  PROCESSING 

• ANY  SALEABLE  END  ITEM 


In  my  opinion,  Hughes  management  level  participa- 
tion is  adequate  as  constituted  in  the  progressive 
acceptance  concept:  the  level  of  participation 
increases  as  you  progress  toward  final  selloff  at 
the  expense  of  continuity. 


Again  in  my  opinion,  continuous  and  joint  industry/ 
Government  participation  is  desirable  and  favored 
over  totally  independent  review  teams  even  though 
participation  by  independent  members  has  some 
benefits. 

I urge  each  of  you  to  consider  adopting  a concept 
like  progressive  acceptance. 


Test  Plan  Review  Board 

While  not  associated  directly  with  progressive 
acceptance,  the  idea  of  more  effective  testing 
introduced  earlier  deserves  at  least  one  chart. 

Each  of  us  in  this  room  at  one  time  or  another  has 
probably  had  occasion  to  conduct  or  assist  in  fail- 
ure analyses. 

That  trail  has  often  lead  to  a dead  end  because 
parameter  N was  tested  in  a different  manner  in 
test  A than  in  test  B,  or  parameters  N and  M,  both 
required  to  meet  requirement  R,  were  tested  under 
different  conditions  in  two  different  test 
configurations. 

As  a result,  Hughes  has  instituted  a practice  we 
are  continuing  to  augment,  which  from  an  overview 
level  examines  how  each  testable  parameter,  every 
time  it  is  measured  throughout  the  production  cycle, 
is  tested,  and  its  data  recorded  and  analyzed  to 
assure  that  parameter  traceability  is  applicable. 
This  permits  in-spec  trend  analysis  - a more  power- 
ful and  sensitive  tool  than  the  traditional  go/no- 
go  criterion. 


• MORE  EFFECTIVE  REVIEWS  - NOT  MORE  TYPES  OF  REVIEWS 

• MORE  EFFECTIVE  FUNCTIONAL  TEST  PARAMETER  TRACEABILITY 

• CONTRACTOR  MANAGEMENT  LEVEL  PARTICIPATION 

• ESCALATE  LEVEL  AS  YOU  PROGRESS  TOWARD  LAUNCH/SELLOFF 

• KEEP  INFORMED  EVEN  IF  NOT  PARTICIPATING 

• INDUSTRY/GOVERNMENT  MIX 

• CONTINUOUS 

• ESCALATE  LEVEL  AS  TIME  PROGRESSES  PARALLELING  CONTRACTOR 
MANAGEMENT  ESCALATION 

• INOEPENOENT  REVIEW  TEAMS 

• TOTALLY  INDEPENDENT  - NOT  COST  EFFECTIVE 

• ■SOME  INDEPENDENT  BOARD  MEMBERS  BENEFICIAL  IF  SKILLED 

AND  ACTIVE 


GOAL 

• IN  SPEC  TREND  ANALYSIS 

• TRACEABILITY 
TECHNIQUE 

• ASSURE  TESTABLE  PARAMETER  TRACEABILITY  FROM  ONE  TEST 
PHASE  TO  EVERY  OTHER  TEST  PHASE 

PLAN 

• CONDUCT  SYSTEM  LEVEL  INFORMAL  REVIEW  OF  EACH  TEST 
PARAMETER  EVERY  TIME  THAT  PARAMETER  IS  TESTED  TO 
ASSURE  TEST  MEASUREMENT  UNIFORMITY.  TEST  DATA 
COLLECTION  CONSISTENCY.  AND  DATA  ANALYSIS  COMPATIBILITY 


Summary 

In  summary,  we  need  more  effective  reviews  rather 
than  more  types  of  reviews:  we  need  more  effective 
testable  parameter  traceability. 
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THE  ROLE  OF  CONTINUOUS  INTERNAL  AUDITS 
IN  MISSION  ASSURANCE 

Mr.  W.H.  Shaw 

Assurance  Audit  Manager,  TRW 


The  workshop  is  considering  various  levels  of 
discrete  management  reviews.  This  discussion 
summarizes  a number  of  contractual  and  internal 
management  review  functions  which  have  been  found 
to  work  well  at  TRW.  However,  undergirding  the 
effectiveness  of  the  discrete  reviews  are  two  con- 
tinuous review  processes.  The  inherent  responsi- 
bility of  each  manager  for  the  performance  of  his 
people,  and  the  performance  of  audits  to  surface 
faults  for  manager  attention  and  action.  Audits 
are  blended  into  a comprehensive  program  at  TRW, 
shaped  by  Group-level  policy.  Direction  of  audit 
functions  (e.g.,  financial,  engineering,  assur- 
ance, etc.)  is  assigned  to  responsible  vice- 
presidents.  This  discussion  reviews  the  develop- 
ment of  the  Assurance  Audit  Program  in  the  context 
of  Mission  Assurance  enhancement. 


In  a company  under  contract  for  a number  of  major 
projects,  there  is  opportunity  for  transfer  of 
experience.  On  the  other  hand,  one  must  guard 
against  inconsistency.  This  tendency  arises  from 
diversity  of  experience  of  people,  both  customer 
and  contractor.  Inconsistency  causes  problems  in 
transferring  people  between  projects,  in  handling 
of  common  facilities  and  systems,  and  in  communi- 
cation with  central  management,  with  central  skill 
centers,  and  with  customers.  Nonstandard  systems 
should  be  resisted  during  the  proposal  and  contract 
negotiation  process.  Internal  management  reviews 
should  produce  lessons-learned  data,  and  the  best 
experience  should  be  codified  in  policy,  with 
management  insistence  thereafter  that  such  stand- 
ards be  followed,  or  uniformly  modified  under 
control  as  experience  continues.  System  reviews 
or  audits  are  a means  for  both  enhancing  cross-feed 
of  the  best  operating  ideas  among  projects  (which 
otherwise  tend  to  be  insular),  and  enforcing  the 
management  resolve  to  operate  with  uniformity. 
Uniform  operating  systems  need  not  stifle  creativ- 
ity, but  rather  should  provide  a stable  routine  to 
allow  accomplishing  the  ordinary  tasks  with  least 
effort  and  error  while  affording  a resiliant  base 
for  coping  intelligently  with  the  extraordinary. 
Uniform  systems  and  procedures  also  provide  a 
standard  against  which  reviews  (audits,  evalua- 
tions) may  be  efficiently  conducted. 

The  TRW  Assurance  Audit  Program  is  under  the 
direction  of  a senior  manager  reporting  to  the 
Vice-President  and  Director  of  Product  Assurance. 
Its  scope  includes  all  the  assurance  disciplines: 
QA,  Configuration  Management,  Reliability,  and 
System  Safety.  The  Assurance  Audit  Office  (AAO)  is 
staffed  by  senior  specialists  who  examine  operating 
systems  for  adequacy  and  existence  of  defining 
policy.  Compliance  audits  are  conducted  in  func- 
tional areas  by  Quality  engineers  under  the  general 
supervision  of  the  AAO  but  assigned  to  the  staffs 
of  the  Division  PA  managers.  Special  audits  are 
also  conducted  at  project  request  to  investigate 
specific  problems  or  concerns.  These  have  been 
applied  extensively  in  software  projects. 


The  discussion  describes  the  audit  program  planning 
process  and  the  techniques  for  conduct  of  audits. 


Topical  emphasis  has  been: 

Operating  Systems  (Including  Test)  3551 
Defect  Control  35% 
Configuration  Management  18% 
Documentation  Systems  12% 


Examples  of  the  topics  covered  and  scope  are  given 
and  short  case  histories  are  reviewed  to  illustrate 
the  effectiveness  of  the  program.  The  goal  of  the 
program  is  to  cause  managers  to  make  their  operat- 
ing systems  work  better.  The  indirect  objective, 
in  the  context  of  this  Workshop,  then,  is  more 
effective  project  reviews  as  a result  of  fewer 
record  discrepancies  and  lesser  volume  of  paper 
describing  the  occurrence  and  correction  of  hard- 
ware mistakes. 

The  Role  of  Continuous  Internal  Audits  in  Mission 
Assurance 


• Value  of  reviews  is  the  preparation  they 
force 

• Undergirded  by  continuous  processes 

• Manager  responsibility  for  performance 
of  his  people 

• Audit  to  expose  faults  for  manager 
action 

• Audit  Goal: 

• Cause  managers  to  make  systems  work 
better 

• Indirect  objective: 

• More  effective  project  reviews 

• Fewer  record  discrepancies 

• Fewer  hardware  faults  and  related 
paper 

• Concentration  on  technical  content 
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TRW  Audit  Program 

• Group  level  policy 

• Functions  assigned  to  responsible  V.P.S. 

• Examples:  Financial 

Management  Systems 

Materiel 

Engineering 

• Assurance  audits  under  director  - product 
assurance 
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Assurance  Audit  Program 


TRW  Internal  Reviews 


• Senior  Manager  reporting  to  VP/Director 

PA 

• System  Audits  - Exi stance,  Adequacy 

• Senior  Specialists 

• All  Assurance  Disciplines 

• Includes  cost  considerations 

• Functional  area  audits  - compliance 

• Quality  Engineers  assigned  to  division 
PA  manager  staffs 

• Special  audits 

• Specific  problem  investigation 

• Project  request 

• Extensive  use  in  software 

System  Audit  Features 

• Topical  Subjects 

Examples:  Materials  Review  System 

Project  CM  Plan  Implementation 
Test  Procedure  Validity 

• Compares  procedure  consistency  among 

projects 

• Compares  performance  among  divisions 

• Involves  all  levels  of  management 

• Depends  heavily  upon  interviews 

• Work  force  waiting  to  be  heard 

• Important  to  protect  sources 

• Expose  problems,  not  people 

• Reports  restricted  - "Need  to  know" 

i Aggressive  follow-up 

Audit  Conduct  and  Results 

• Topic  Selection 

• Data  system  trouble  indicators 

• New  systems  are  suspect 

• Priority  recommendations  from  division 
PA  managers 

• Quarterly  planning 

• Written  task  plan 

• Topic  emphasis 

• Operating  systems  (including  test)  35% 

• Defect  control  35% 

• Configuration  management  18% 

• Documentation  systems  12% 

• Results  and  observations 

• Insufficient  training  in  new  systems 

• Capable  work  force  crutches  system 
gaps 

• First  level  management  defensive 

• Cross-fertilization 

e Advisory  source  for  new  system  design 
and  discrete  reviews 


Project-Oriented 


Monthly  management  meeting  - 
Project  manager  review 
Project  staff 
Test  program  review 
Launch  readiness  review 
Software  division 


major  projects 

monthly 

weekly 

test  plan 

weekly 

timeshare  data 


TRW  Internal  Reviews 
Institutional 

Test  discrepancy  reports  - weekly 

In-process  corrective  action  boards  - weekly 

Corrective  action  boards  - monthly 

Manufacturer  evaluation  control  - standing 

board 

Configuration  management  - standing 

organization 

Software  policies 
Audit  program 


Review  Lessons  Learded 

• Timing  important 

• Too  early  - insufficient  data 

• Too  late  - opportunity  for  action  lost 

• Management  level 

• Decision  oriented 

• Level  necessary  to  act 

• Overkill  produces  ineffective  review 

• Emphasis  if  resources  limited 

• Front  load  - requirements  definition 
system  engineering 

• Software  studies  - 60  code  faults 
requirements  and  design 

• Preparation  essential 

• Necessary  data 

• Homework 

• Discussion  of  alternative 

• Principals  understand  subject 

t Objectives 

• Force  communication 

• Focus  attention  on  vaguely  acknowledge 
problems 

• Enforce  milestones 


Government  Review  Process 

• Issues  re  institutional  reviews 

• Leader  has  pressures/problems 

• Lead  to  sweeping  conclusions  from 
insufficient  data 
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• Solutions 

• Team  Preparation 

• Issues 

• Objectives 

• Approach 

• Assignments 

• Ground  rules 

• Contractor  interface  team  - help  plan 

• Couple  senior  contractor  people 

• Review  manager's  staff 

• Morning  team  sessions 

• Screen  results  with  review  manager 

Multi-Project  Environment 

• Opportunity  - transfer  of  experience 

• Risk  - inconsistency 

• Wastes  effort 

• Problems  - people  transfer 

- common  facilities 

- communication 

• Resist  during  proposal  and  negotiation 

e Management  reviews  - contribute  to 
solutions 

• Lessons  learned 

• Codify  best  experience 

• Insist  upon  conformance 

• System  audits 

• Discover  needless  inconsistencies 

• Pinpoint  areas  of  confusion  or  cost 

• Identify  policy  gaps 

• Cross  fertilize 

Review  Team  Mix 

• Risks 

• May  hide  problems 

• Benefits 

• Avoid  false  starts 

• Prevent  intemperate  or  trivial 
conclusions 

• Prevent  contractor  resources  allocated 
to  wrong  problems 

• Get  fixes  rolling  earlier  on  real 
problems 

• Near-term  objectives  must  be  sufficiently 

common 


INTERNAL  REVIEW  - A MANAGEMENT  CONTROL 
Walter  L.  Finch, 

Lockheed  Missiles  & Space  Company,  Inc. 


We  all  know  that  a successful  mission  does  not 
necessarily  demonstrate  an  adequate  management  con- 
trol system,  let  alone  one  which  is  optimum.  How- 
ever, continued  success  reasonably  leads  one  to 
believe  that  the  management  controls  are  good. 
Unfortunately  we  tend  to  ignore  the  first  and  to 
over-believe  the  second. 

In  the  event  of  flight  failure  there  is  an  immedi- 
ate reaction  - perhaps  over  reaction!  Find  the 
guilty  party/equipment  and  take  action  to  assure 
that  one  won't  ever  happen  again!  This  includes 
an  instant  conclusion  that  something  is  wrong  with 
the  program  controls  to  allow  the  failure  to  occur. 
Any  area  which  could  have  created  failure  (or 
allowed  it)  is  magnified  - perhaps  out  of  propor- 
tion - and  corrective  action  taken  to  assure  pre- 
vention in  the  future.  This  could  lead  one  to 
conclude  that  our  current  Government/Industry 
review  process  is  likely  excessive  - even  to  the 
point  it  has  become  counter-productive.  As  an 
example,  extra  controls  (that  are  complex  and  time 
consuming)  to  provide  protection  against  exercise 
of  poor  judgment  (like  failure  to  follow  instruc- 
tions) when  the  likelihood  of  a repeat  is  unlikely 
and  the  added  complexity  does  not  prevent  bad  judg- 
ment, only  adds  cost.  Please  don't  jump  to  the 
conclusion  I am  suggesting  flight  failure  analysis 
and  corrective  action  are  not  necessary  and  proper  - 
only  that  the  resulting  actions  are  often  not  cost- 
effective. 

The  analysis  produced  long  before  flight,  as  a 
result  of  prior  data,  clearly  shows  the  probability 
of  success  is  less  than  one.  The  problem  is  that 
a 100  percent  reliability  is  what  is  really  wanted, 
no  matter  what  is  contracted  for,  and  in  the  real 
world  comes  to  be  what  is  expected  - probably  all 
the  way  to  the  Pentagon.  The  point  being  that  this 
generates  emotional  conditions  which  generate  emo- 
tional decisions.  And  in  the  main  it  has  been  like 
heresy  (perhaps  a more  familiar  term  would  be  non- 
responsive)  to  suggest  anything  like  elimination  of 
any  formal  Government  requirements/controls.  This 
applies  specifically  to  standards  imposed  contrac- 
tually which  are  supposed  to  be  used  as  guides  and 
therefore  tailored  to  program  needs  by  judgment. 

This  application  of  judgment  varies  widely  from 
program  to  program  - much  beyond  that  one  could 
expect  as  a result  of  differences  in  program  com- 
plexity, funding,  importance,  state-of-the-art, 
etc.  It  seems  to  be  quite  dependent  on  the  ability 
of  the  Government  Program  Manager.  He  must  under- 
stand and  support  implementation  of  real  needs  and 
fend  off  the  "standards  zealots"  who  believe  that 
all  contractors  are  wild  men  and  the  specifications/ 
standards  must  be  enforced  to  the  Nth  degree  to 
safeguard  the  Government's  interests.  Good  busi- 
ness practice  dictates  well  defined  contractual 
agreements  but  good  judgment  in  contractual  appli- 
cation of  standards  is  cost-effective  and  can  have 
a dramatic  effect  in  taming  that  "wild  man". 


Let's  take  a look  at  our  current  Space  Systems 
Division  management  controls  to  see  if  they  will 
shed  any  light  on  the  question  of  an  optimum  for 
Government/Industry.  And  I would  like  to  alert  you 
right  now  that  with  the  time  allowed  there  is  no 


way  to  give  a detailed  description  - so  a brief 
overview  followed  by  in-depth  review  of  some  key 
elements  will  be  my  modus  operandi. 

Our  system  of  controls  is  probably  typical  of  other 
major  prime  contractors  and  is  the  result  of  many 
years  of  experience.  It  includes  the  advice  and 
consel  of  many  capable  and  dedicated  Government 
personnel  - both  military  and  civilian.  Review/ 
audit  is  accomplished  by  the  General  Manager, 

Program  Manager,  System  Engineering,  Reliability 
Engineering,  Product  Assurance,  Design  Engineering, 
Functional  Management  of  all  supporting  organiza- 
tions, and  by  the  Customer.  Customer  actions  are 
implemented  by  members  of  the  Air  Force  Program 
Office,  Air  Force  Plant  Representative,  Aerospace 
Corporation,  and  by  other  experts  hired  by  the 
Customer  for  that  purpose.  For  our  discussion  I 
will  group  all  of  these  under  one  category  - Cus- 
tomer. The  list  is  long  and  yet  does  not  include 
the  many  working  level  activities  which  constitute 
an  independent  audit. 

The  Program  Review  is  handled  by  the  Program 
Manager  and  highlights  status  - problem  areas  - and 
open  action  item  assignments.  Attendance  by  Cus- 
tomer varies  - in  some  programs  they  do  not  attend 
but  get  a special  briefing  on  the  results  plus 
notification  of  any  major  problem  areas.  Usually 
there  will  be  some  members  of  General  Management 
in  attendance.  The  Quarterly  General  Manager 
Review  covers  the  same  type  data  but  includes  all 
his  staff  which  gets  across  Program  information 
interchange  and  opportunity  for  assistance  in  pro- 
blem solution.  The  same  holds  true  for  the  weekly 
Chief  Engineer's  meeting  where  technical  problems 
and  significant  accomplishments  are  discussed.  The 
Vide  President  and  Assistant  General  Manager  of  the 
Division  attends  and  participates  in  these  meetings. 
The  Top  Ten  is  self-explanatory  and  is  issued  by 
the  Program  Manager  to  all  levels  of  management 
appropriate  - including  the  General  Manager.  Con- 
tract Program  Reviews  usually  are  at  milestones  like 
System  Requirements  Review,  Preliminary  Design 
Review,  and  Final  Design  Review  with  the  format 
defined  by  the  contract.  Parts  control  is  a funda- 
mental requirement  for  mission  success  - the  subject 
of  another  workshop.  Let  me  just  mention  here  that 
configuration  analysis  is  a System  Design  Review  at 
the  part  level  - including  a FMEA,  and  a determina- 
tion of  the  criticality  of  the  controls  of  the 
materials,  processes,  and  assembly  techniques  used, 
plus  evaluation  of  the  inherent  design  reliability 
and  effective  screening  techniques  to  identify  and 
eliminate  parts  manufactured  with  incipient  failure 
modes. 

Design  review  is  another  fundamental  action  needed 
to  assure  mission  success  - and  the  day-to-day 
activity  which  preceeds  the  formal  review  meeting 
is  a key  element  - particularly  if  we  are  to  avoid 
an  undue  dependence  on  the  review  meeting  to 
accomplish  the  design  - rather  than  an  effort  to 
verify  the  adequacy  of  what  has  been  done  and, 
hopefully,  to  provide  some  fine  tuning.  Customer 
participation  varies  widely  - as  does  the  opinion 
in  regard  to  effectiveness  of  this  participation. 

It  certainly  helps  to  generate  confidence;  cut  it 
can  inhibit  the  participants.  The  ability  of  the 
individual  representing  the  Customer  to  create  an 
atmosphere  where  serious  disagreements  are  likely 
to  be  aired  is  probably  the  key.  The  review  should 
be  hardware  oriented  and  not  an  educational  pro- 
cess for  senior  levels  of  management,  and  selection 
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of  the  Review  Team  (including  assignment  of  the 
chairman)  is  a critical  element  which  should  have 
the  active  involvement  of  the  Program  Manager.  The 
reason  for  saying  so  much  about  Design  Review  in 
this  workshop  - when  there  is  another  one  on  design 
which  will  surely  cover  it  in  more  detail  - is  that 
I believe  when  it  is  done  right  the  need  for 
special  audits  later  is  significantly  reduced  and 
will  improve  their  effectiveness  when  conducted. 

Get  the  best  people  available!  Leader  key  element! 

The  CCB  (Configuration  Control  Board)  is  an  adjunct 
to  the  design  review  process  and  needs  no  addi- 
tional comment.  The  system  safety  activity  is  an 
area  I don't  intend  to  cover  in  any  detail  today. 
Suffice  to  say  it  has  been  improving  steadily  and 
had  made  significant  gains  in  the  past  five  years. 
However,  the  cost  of  accidents  has  also  increased 
significantly  so  that  continued  emphasis  is  needed 
to  drive  down  the  accident  rate.  And  efforts  are 
under  way  to  do  so.  One  of  the  most  effective 
measures  to  aid  this  is  to  avoid  panic  work  hour 
schedules  whenever  possible. 

The  Corrective  Action  Review  Board  includes  all 
appropriate  functional  management,  including  the 
General  Manager,  and  Is  convened  whenever  necessary 
to  obtain  action.  This  applies  to  the  Senior  MRB, 
too  - but  they  both  meet  on  a quarterly  basis  to 
be  kep  up-to-date  in  regard  to  problem  status. 

The  next  two  items  will  be  covered  in  another  work- 
shop and  I will  not  have  any  detailed  comments. 

The  Qualification  Test  and  Acceptance  Test  are  two 
more  key  elements  in  mission  assurance.  It  is 
important  they  be  properly  defined,  the  data  be 
objectively  assessed,  and  that  we  pay  attention  to 
the  results  of  the  data  analysis.  The  Component 
Acceptance  Data  Package  Review,  therefore,  is  also 
a key  element.  The  goal  of  this  should  be  to 
identify,  through  examination  of  the  documentation, 
any  anomaly  which  could  adversely  affect  the  flight 
worthiness  of  the  unit  being  reviewed.  Since  each 
unit  is  judged  by  the  team  on  the  basis  of  the 
results  of  the  review,  there  should  be  no  absolute 
acceptance  or  rejection  requirements.  However,  if 
the  team's  decision  is  to  reject  the  unit,  each 
unacceptable  finding  is  clearly  documented.  In 
addition,  specific  direction  (or  a date  for  that 
direction)  is  developed  by  the  team  to  establish 
the  minimum  acceptable  rework,  repair,  or  retest. 

Let's  continue  with  First  and  Nth  Article 
inspection  and/or  audit.  This,  of  course,  is 
related  to  programs  where  there  is  more  than  one 
unit  planned  to  be  manufactured  - but  it  can  also 
be  a useful  technique  for  single  unit  builds  - at 
least  it  is  worth  consideration.  It  is  another 
key  element  in  mission  assurance. 

If  money  were  no  object  it  would  be  preferable  to 
schedule  two  of  these  audits.  The  first  would 
establish  a baseline  by  verifying  that  the  first 
production  unit  conforms  to  the  released  engineer- 
ing drawings,  and  that  adequate  controls,  proced- 
ures, specifications,  etc.,  exist  to  assure  that 
subsequent  production  would  continue  to  have  the 
same  configuration.  The  Nth  would  then  be  picked 
in  some  random  manner  to  verify  the  configuration. 
However,  money  is  an  object  as  resources  are 
limited.  Therefore,  where  there  can  only  be  one 
it  is  best  done  on  the  First  Flight  Article. 


Regardless  of  which  unit  is  selected,  maximum 
visibility  of  the  hardware  is  the  major  considera- 
tion for  when  the  audit  is  scheduled.  The  primary 
interest  of  the  team  is  examination  of  the  hardware. 
It  is  not  to  be  a "dog  and  pony  show"  or  even  a 
"presentation". 

Another  facet  of  the  hardware  audit  is  the  potential 
for  finding  better  ways  of  building  the  unit. 
Obviously,  the  direct  approach  is  when  a team  mem- 
ber has  assisted  in  the  resolution  of  similar  pro- 
blems at  other  locations.  Indirectly,  and  possibly 
of  even  greater  value,  is  when  the  team  acts  as  a 
catalyst  to  promote  dialogue  between  design  and 
shop  personnel.  The  presence  of  a "third  party" 
can  stimulate  the  flow  of  ideas  which  may  highlight 
long-standing  production  hang-ups.  The  shop  person- 
nel often  have  a "better  idea",  and  the  climate  of 
the  hardware  audit  can  encourage  its  implementation. 

Although  the  examination  starts  with  the  hardware, 
and  much  attention  is  given  to  workmanship  and 
evidence  of  quality  assurance  inspection,  questions 
relating  to  procedures  and  processes  arise  very 
rapidly.  Since  the  goal  is  to  achieve  repeatable 
reliability,  it  isn't  enough  to  find  the  hardware 
in  excellent  condition  — that  may  only  reflect 
the  skill  of  an  operator  who  may  retire  or  be  laid 
off  before  the  next  unit  is  built. 

As  an  illustration  of  this,  let  me  relate  an 
incident  observed  at  one  of  our  suppliers.  We  were 
witnessing  a critical  welding  process  and  the  oper- 
ator was  handing  us  the  small  pieces  for  examination 
as  he  completed  them.  To  a person  familiar  with 
welding,  these  parts  were  a thing  of  beauty  — 
penetration  was  perfection,  splatter  was  non- 
existent. They  couldn't  have  been  better  if  they 
had  been  hand-picked  for  display  purposes.  One 
member  of  the  team,  familiar  with  the  set-up  and 
controls  for  such  machinery,  asked  the  operator 
how  he  knew  what  settings  to  use.  (There  were 
about  four  separate  controls  to  adjust  on  this 
machine  depending  on  the  job  to  be  performed.) 

The  operator  indicated  that  a procedure  was  avail- 
able and  one  was  brought  in  very  quickly.  However, 
none  of  the  variable  controls  were  set  in  accord- 
ance with  the  procedure.  When  asked  what  would 
happen  if  the  settings  were  made  per  the  pro- 
cedure, the  operator  replied,  "Here,  I'll  show 
you."  He  thanged  the  settings  and  welded  several 
parts  which  had  been  scrapped  for  other  reasons. 

The  results  were  astonishing!  The  welding  literally 
blew  some  of  the  delicate  parts  into  fragments  and 
in  no  way  was  the  end  result  acceptable.  This 
skilled  operator  had  been  compensating  for  the 
aging  of  the  machine  for  several  years.  Another 
operator,  unfamiliar  with  the  process,  wouldn't 
have  a ghost  of  a chance  to  build  good  hardware 
using  the  existing  procedure. 

I can  almost  hear  some  of  you  saying  right  now, 
"That's  not  such  a big  deal!.  It  would  easily  be 
caught  by  Inspection."  That's  probably  true  for 
the  illustration;  but  might  not  be  with  a different 
operator  who  also  made  his  own  settings  which 
resulted  In  hardware  with  an  incipient  failure  mode. 
Think  a little!  I suspect  each  of  you  could 
Identify  some  operation  In  your  plant  where  the 
deviation  from  the  procedure  would  produce  a subtle, 
but  possibly  critical  anomaly  that  might  not  be 
easily  detected.  That  is  one  thing  the  Nth 
Article  Audit  seeks  to  Identify  and  eliminate. 


Another  example  is  related  to  manufacture  of  a 
multilayer  PCB.  A mask  for  a contact  had  slipped 
after  wash  and  etch  resulting  in  a trace  width 
being  inadvertently  narrowed  - but  with  enough 
width  to  be  possible  to  pass  acceptance  tests  and 
represent  an  incipient  failure  mode  in  extended 
flight  use. 

This  brings  me  to  the  last  item  on  this  chart. 

The  hardware  audits  are  intended  to  be  performed 
down  to  the  nitty-gritty.  A flight  failure  due  to 
a minor  problem  is  no  different  than  one  from  a 
major  anomaly.  Looking  at  it  another  way  --  it  is 
proper  that  the  only  findings  of  an  audit  are  minor 
ones.  After  all,  at  the  time  a hardware  audit  is 
conducted,  the  best  efforts  of  skilled  personnel 
have  been  applied  to  the  design  and  fabrication  of 
the  hardware  for  a year  or  more.  In  addition, 
there  have  been  numerous  hours  of  technical  inter- 
change with  the  Responsible  Equipment  Engineer  as 
well  as  a couple  of  Design  Reviews.  It  would  be 
a blow  to  all  our  prides  if  something  major  had 
been  overlooked. 

I have  already  indicated  personnel  selection  to  be 
a key  element  in  Design  Review.  This  chart  shows 
the  organizations  represented  on  the  teams 
involved  with  the  last  three  types  of  reviews  dis- 
cussed. For  an  adequate  review,  it  is  necessary 
to  balance  product  familiarity  with  objectivity/ 
independence.  This  appears  to  be  accomplished  and 
at  the  same  time  there  is  consistency  of  composition. 


The  running  sell  is  the  final  key  element  in  our 
system  to  provide  mission  assurance.  The  chart  is 
self-explanatory  and  the  data  is  used  at  DD-250  and 
the  Launch  Readiness  Reviews.  As  a matter  of  fact, 
having  done  those  tasks  well  as  described  earlier, 
the  DD-250  and  Launch  Readiness  Review  activities 
should  be  relatively  simple  and  routine. 

I would  like  now  to  provide  you  my  summary  of  the 
significant  conclusions  one  might  reach  based  on 
experience  weighted  by  the  data  presented.  This 
includes  some  consensus  of  working  level  and  man- 
agement personnel,  of  both  functional  and  program 
organizations. 

Reviews  or  audits  as  an  in-line  function  generally 
are  more  effective  than  special  audits.  There  are 
several  reasons  for  this  and  they  mostly  relate  to 
the  key  elements  for  any  successful  audit.  They 
are: 

1.  Proper  timing  so  that  (a)  results  are 
meaningful,  (b)  work  load  of  auditees  is 
not  so  heavy  as  to  create  an  additional 
hazard  from  interruptions  by  the 
auditors. 

2.  Sufficient  effort  to  make  careful 
selection  of  audit  team. 

3.  Support  of  Senior  Management  to  assure 
objectivity/independence  of  audit  --  and 
action  on  findings  where  necessary. 

4.  Good  definition  of  scope  and  objectives 
of  the  audit  - plus  any  special  ground 
rules. 

5.  Pre-planning  to  define  method/manner  of 
conduct  of  audit. 


6.  Clear  team  member  assignments. 

7.  Chairman  must  be  in  charge. 

8.  Must  be  free  of  State  findings  — what- 
ever  they  are  — with  basis.  Resul ts 
should  go  first  to  manager  responsible 
with  time  to  evaluate/discuss/refute/ 
take  action/decide  on  action,  etc.,  prior 
to  any  inputs  at  higher  level  within 
some  pre-established  time  limits. 

9.  Make  sure  the  audit  team  has  no  built-in 
requirement  to  find  problems  in  order  to 
justify  their  selection  on  the  team  or  to 
demonstrate  the  omnipotence  of  the 
'auditors'. 

10.  Audit  team  needs  to  have  product  know- 
ledge and  independence. 

11.  Recognition  by  team  and  auditees  that 
review  needs  to  be  in  "nit-picking" 
detail  - that  some  of  the  findings 'will 
likely  appear  minor  in  nature  - and  that 
is  not  bad  since  one  would  not  expect  to 
find  major  problems  after  the  very  best 
people  had  "done  their  thing". 

12.  Remember  that  state-of-the-art  has  been 
carefully  reviewed/analyzed/audited,  etc., 
by  almost  everyone  any  way  connected  and 
usually  by  the  most  capable  personnel 
from  both  technical  and  management  dis- 
ciplines - while  non-state-of-the-art 
generally  gets  perfunctory  review/ 
manpower/ time/fundi ng , etc.,  and  more 
often  than  not  is  the  item  which  really 
gets  you  in  trouble. 

It  should  be  apparent  from  this  list  that  it  is 
much  more  difficult  to  satisfy  the  needs  for  a 
"special " audit.  The  timing  is  usually  wrong: 
with  work  load  too  high;  insufficient  time  for 
anything  but  a surface  check;  best  team  member 
selections  not  available;  little  or  no  time  avail- 
able for  action  unless  a fault  were  identified 
which  clearly  spelled  out  doom  for  the  mission  - 
and  these  are  not  very  likely. 

It  also  appears  that  the  system  in  being  has 
sufficient  controls  - independence  - and  management 
attention  to  provide  for  mission  assurance.  Tiger 
teams  and  special  audits  over-done  get  looked  on 
as  a panacea  and  I believe  motivate  people  sub- 
consciously  to  do  less  thorough  work  than  they 
would  otherwise.  In  addition,  they  seem  to 
encourage  every  agency  to  get  into  the  act  so  that 
busy  people  don't  have  time  to  do  their  jots  - with 
the  possibility  that  this  overkill  is  a counter- 
productive action.  On  a running  basis  it  would 
appear  that  adequate  independence  is  provided 
through  the  Chief  Systems  Engineer  and  Reliability 
Engineering  functions.  For  special  audits,  how- 
ever, one  should  go  outside  the  program/function 
being  audited  (either  within  LMSC  or  A or  AF)  for 
chairman  and  at  least  some  of  the  team  members. 

On  the  spot  Customer  Program  Office  Representatives 
appears  to  be  an  effective  method  of  operation  if 
they  are  provided  authority  to  make  decisions  and 
are  truly  knowledgeable  in  the  discipline.  Other- 
wise, they  are  just  another  layer  of  reviewers  who 
provide  data  for  top  side  decisions.  Any  outside 
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talent  used  should  be  selected  on  the  basis  of  the 
experience/expertise  they  add;  with  a caution  to 
make  sure  their  experience  is  really  relevant  to 
our  products  and  they  become  familiar  with  the 
equipment  prior  to  participation  in  reviews/audits 


SSD  REVIEW/ AUDIT  LIST 


WEEKLY 

QUARTERLY 

WEEKLY 


GENERAL  MANAGER  REVIEW 


TOP  TEN  PROBLEM  REPORT  _ 

CONTRACT  PROGRAM  REVIEW  _ 

CHIEF  ENGINEERS  MEETING 

(V.P.  & ASST.  GEN.  MGR.  ATTENDS  I 
PARTS  CONTROL  BOARD  IAS  REQUIRED  I 
GIDEP  ALERT  SYSTEM 
CONFIGURATION  ANALYSIS 
PREFERRED  PARTS  LIST/ PROGRAM 


MAJOR  MILESTONES 


The  system  now  in  being,  with  some  minor  tuning  and 
attention  to  detail,  will  provide  that  optimum 
system  for  mission  assurance. 


WEEKLY 


REGULAR 


SCHEDULE 


CONTROLLED  PARTS  LIST 


START 

SCHEDULE 


DESTRUCTIVE  PHYSICAL  ANALYSIS 


DESIGN  REVIEW 


THINGS  AREN'T  ALWAYS  WHAT  THEY  SEEM 


SCHEDULE 


CONFIGURATION  CONTROL  BOARD 
SAFETY  REVIEW  BOARD  - 
PROGRAM  SYSTEM  SAFETY  COMMITTEE 
CORRECTIVE  ACTION  REVIEW  BOARD 
SENIOR  MATERIAL  REVIEW  BOARD  _ 
AFQA  STATUS  REVIEWS 
QUALITY  SYSTEMS  AUDIT  _ 


AS  REQUIRED 
MAJOR  MILESTONE 
WEEKLY/MONTHLY 


AS  REQUIRED 
QUARTERLY 


CONTINUOUS 


CONFIGURATION  ANALYSIS 


PERFORMANCE  HISTORY 
DEVELOPMENT 
QUALIFICATION 
ACCEPTANCE 
FLIGHT 


PROCESS  ANALYSIS 
IDENTIFY  CRITICAL 
PROCESSES 

ELIMINATE  AMBIGUITIES 
GENERATE  DETAILED. 
ACCURATE  FLOW  CHART 


MATERIAL  ANALYSIS 

SPECIFIC  IDENTIFICATION 
MFGRS  DESIGNATIONS 
CHEMICAL  OR  PHYSICAL 
ANALYSIS,  IF  REQUIRED 


CONFIGURATION 


DISASSEMBLY 
EXPLODED  VIEW 
OF  ASSEMBLY 


FAILURE  MODE 
AND 

EFFECTS  ANALYSIS 


DESIGN  REVIEWS 


• PRELIMINARY  (CONCEPT) 


• AFTER  DESIGN  CONCEPT,  BEFORE  DETAIL  DESIGN 

• REVIEW  INTERFACE  DEFINITION 

• REVIEW  AND  EVALUATE  DESIGN  CONCEPTS 

• COMPARISON  OF  DESIGN  APPROACH  WITH  REQUIREMENTS 

• REVIEW  TRADE-OFFS 


• CRITICAL  (DETAIL) 


• 90  PERCENT  DRAWING  REUASE 

• VERIFY  PRODUCIBILITY  OF  DESIGN 

• EVALUATE  TEST  PROGRAMS 

• VERIFY  DESIGN  VS  REQUIREMENTS 

• ESTABLISH  READINESS  TO  FABRICATEMEST  HARDWARE 


SSD  REVIEW/ AUDIT  LIST 


QUALIFICATION  TEST 

I PRIOR  TO  FINAL  DESIGN  REVIEW) 

SOURCE  INSPECTION 

ACCEPTANCE  TEST 

COMPOMNT  ACCEPT.  DATA  PKG.  REVIEW 

COMPLIANCE  VERIFICATION  MATRIX  

/TECHNICAL  PERFORMANCE  MEASUREMENTS) 
INTERFACE  CONTROL  WORKING  GROUP  MEETINGS 

MANUFACTURING  PROGRAM  REVIEWS 

I ST  ANO  NTH  ARTICLE  INSPECTIONS /AUDITS 

SPACE  VEHICLE  CUSTOMER  BUY-OFF  RUNNING  SELL 


SCHEOUlf 


• DESIGN  COMPLETE /QUALIFICATION  REPORT  AVAILABLE 

• VERIFY  HARDWARE  READINESS 

• REVIEW  MANUFACTURING  PROCESS 

• EVALUATE  RELIABILITY  ANALYSIS 

• REVIEW  QUAL  TEST  RESULTS 


SCHEDULE 


SCHEDULE 


SCHEDULE 


SCHEDULE 

SCHEDULE 


READINESS  REVIEW 

SCHEDULE 

SPECIAL  AUDITS  

SCHEDULE 

r 


PROGRAM  QUALITY  ASSURANCE  ACTIVITIES 


ACCEPTANCE  DATA  REVIEW 


• ACCEPTANCE  TEST  COMPUTE 

• PRODUCT  ASSURANCE  SUPPLIER  REPRESENTATIVE  (PASR) 
REVIEW  COMPUTE 


HARDWARE  AUDITS 


• NORMALLY,  ONE  SCHEDULED  PER  BLOCK 

• MAXIMUM  VISIBILITY  OF  HARDWARE 

- AT  LEAST  ONE  COMPLETED  UNIT 

- COMPLETE  SET  OF  SUBASSEMBLIES 

• NOT  A " DOG  AND  PONY"  SHOW 

• EXAMINE  FOR: 

- WORKMANSHIP 

- INSPECTION 

- PROCEDURES 

- PROCESSES 


• PAINSTAKING  EXAMINATION  OF: 

-BUILD  HISTORY  DOCUMENTATION 

- SUBASSEMBLY  TEST  DATA 

- ACCEPTANCE  TEST  DATA 

- FAILURE  DATA 

-ENGINEERING  CHANGE  INCORPORATION 

• CRITERIA 

- COMPLETENESS  AND  SEQUENCE 
-ACCURACY  AND  UGIBILITY 

-VERIFICATION  OF  PARTS  TRACEABILITY  INFORMATION 
-EVIDENCE  OF  INTERNAL  CONTROL 

• FAILURE  DOCUMENTATION 

- DETAILS  OF  THE  FAILURE 

- TROUBUSHOOTING  AND  ANALYSIS  PROCEDURE 

- RESULTS  OF  FAILURE  VERIFICATION 

- RESULTS  OF  STRESS  ANALYSES 

- REPAIR  INSTRUCTIONS 

- CORRECTIVE  ACTION 

- ADEQUACY  OF  THE  SPECIFIED  RETEST 

• SPECIAL  ATTENTION  FOR  ITEMS  DISPOSITIONED 
"USE-AS-IS” 

• TECHNICAL  ACCEPTANCE 


• GOAL  - REPEATABLE  RELIABILITY 

• NIT-PICKING! 
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TEAM  COMPOSITION 


KEY  ELEMENTS  FOR  SUCCESSFUL  REVIEW /AUDIT 


• DESIGN  REVIEWS 

- CHAIRMAN,  VEHICLE  DESIGN  INTEGRATION 

- RESPONSIBLE  EQUIPMENT  ENGINEER 

- RESPONSIBLE  RELIABILITY  ENGINEER 

- SYSTEMS  ENGINEERING 

- PRODUCT  ASSURANCE 

- SPECIALISTS  AS  REQUIRED,  SPACE  TECHNOLOGY, 

- MFC  SYSTEMS  ENGINEERING,  ETC. 

• N,h  ARTICLE  AUDITS 

- CHAIRMAN,  VEHICLE  DESIGN  INTEGRATION 

- RESPONSIBLE  EQUIPMENT  ENGINEER 

- RESPONSIBLE  RELIABILITY  ENGINEER 

- PRODUCT  ASSURANCE 

- MFC  SYSTEMS  ENGINEERING 

• ACCEPTANCE  DATA  REVIEWS 

- CHAIRMAN,  VEHICLE  DESIGN  INTEGRATION 

- RESPONSIBLE  EQUIPMENT  ENGINEER 

- RESPONSIBLE  RELIABILITY  ENGINEER 

- SYSTEMS  ENGINEERING 

- PRODUCT  ASSURANCE 


SPACE  VEHICLE  CUSTOMER  BUY -OFF  RUNNING  SELL 


• PROPER  TIMING 

• CAREFUL  SELECTION  OF  TEAM 

• SUPPORT  OF  SENIOR  MANAGEMENT 

• GOOD  DEFINITION  OF  SCOPE/OBJECTIVES/GROUND  RULES 

• PRE-PLANNING 

• CLEAR  TEAM  ASSIGNMENTS 

• CHAIRMAN  IN  CHARGE 

• FREE  TO  STATE  FINDINGS 

• NO  BUILT-IN  REQUIREMENT  TO  FIND  PROBLEMS 

• PRODUCT  KNOWLEDGE  AND  INDEPENDENCE 

• ATTENTION  TO  DETAIL 


MANAGEMENT  AND  PROGRAM  REVIEW  PROCESS 


Ur/ 


in Imwsmy 


_ APPROPRIATI 
nrt  • McistoN  uvn 


FOR 

MISSION  ASSURANCt 


• RUNNING  SELL  CONCEPT  ALLOWS  CUSTOMER  TO  INCREMENTALLY 
ACCEPT  THE  SPACE  VEHICLE 

• PROVIDES  CUSTOMER  OPPORTUNITY  TO  ACHIEVE  IN-DEPTH 
EVALUATION 

• CUSTOMER  REVIEW  AND  APPROVAL  POINTS  AFTER  EACH  MAJOR 
TEST  COMPLETION  I.E..  COMPLETION  OF  FINAL  ASSEMBLY, 
AMBIENT  FUNCTIONAL  BASELINE,  ACOUSTIC.  THERMAL  VACUUM. 
FINAL  PREPS  ANO  PREPS  FOR  LAUNCH 

• REVIEW  INCLUDES  DATA  DISCREPANCY  CLOSURE,  VEHICLE 
CONFIGURATION.  STATUS.  ANO  PERFORMANCE  PARAMETERS 

• BEGINS  BUILDING  CONFIDENCE  IN  SPACE  VEHICLE  EARLY 
IN  THE  PROGRAM 


CONTRACTOR'S  APPROACH  TO  INDEPENDENCE  IN 
PROGRAM  REVIEWS 


SAMTEC 


Ben  J.  Wier 

Manager,  Atlas  Programs 
General  Dynamics 


AIL  AS  AND  CENTAUR  PROGRAM  REVIEWS 


MONTHLY 

• GENERAL  MANAGER'S  REVIEW 

• M ERROR  AREURANCE  AJUARTEHLY) 

• yiiAi.rrv  and  workmanship  review  hoard 

• BHtXIRAMS  FLATUS  REVIEW 

on-going 

• qualtty  arriirance  audits 

• MANUFAC1TTRING  IN-l*HOCEES  AUDTT 

• FLIGHT  HARDWARE  I’ERTIFICATEIN  HOARD 

• ENGINEERING  HEVIEW  HOAHII 

• SOFTWARE  REVIEW  HOARD 

• PARTS  DISASSEMBLY  PROGRAM 

• MATERIAL  REVIEW  HOARD 

• MECHANICAL  HARDWARE  DISASSEMH1.Y  AUDT1 


WEEELY 

• TASE  IMPLEMENTATION  BOARD 

• CHANGE  BOARD 

« MANUFACTURING  REVIEW 

• LAUNCH  VEHICLE  RE14ABI1JTY  BOARD 

• PIECE  PARTS  CONTROL  BOARD 

REQUIRED  BY  PROGRAM 

• HEyUDIEMENTS  HEVIEW 

• PRELIMINARY  DESIGN  REVIEW 

• CRITICAL  DESIGN  REVIEW 

• FACTORY  PRODUCT  REVIEW  TEAM 

• FCA  AND  PCA 

• GAR  REVIEW 

• SYSTEMS  READINESS  REVIEW 

• LAUNCH  SITE  PRODUCT  HEVIEW  TEAM 

• LAUNCH  SUPPORT  TEAM 

• PORT  FLIGHT  DATA  REVIEW 


HUNCH  VOUCH  PROCRAMS 
INOIPLNOINI  HIHI*  IIVU  S 


PDR  and  CDR 

• PDR  and  CDR  commit  program  to  design 

• Current  standards  - rigorous  and 
adequate 

• Provide  independent  assessments 

• Formal  presentation 

• Documentation  sent  to  review 
agencies  two  weeks  before  review 

• Formal  minutes  and  action  items 
PDR  and  CDR  Independent  Participation 

GDC 

• Two  levels  of  independence 
SAMSO/Aerospace 

• LV  Engineering  Office  and  Aerospace 

• LV  Operations  Office  and  Aerospace 

• LV  Mission  Assurance  Office  and  Aerospace 


• 6595th  Test  Wing  Operations  and  Aerospace 

• SAMTEC  Range  Operations  and  FEC 

SAilSO  Payload  Office 

• Project  Engineering  and  Aerospace 
Others 

• Payload  Contractor 

• NASA/Lewis 

• NASA/Goddard 

Quality  Assurance  Audit  Team 
t Twelve-man  team 

• Reports  to  Director,  Quality  Assurance 

• All-inclusive  charter 

• In-house  audits 

• Planning 
e Hardware 

• Processes 

• Test 

• Calibration 

• Mission  assurance  requirements 

• Special  Audits 

• Suppl ier  audits 

• Planning 

• Hardware 
a Processes 

• Test 

• Calibration 

Quality  Assurance  Audit  Team  Benefits 

• Detect  and  report  system  deficiencies  to 
management 

• Shop  personnel  and  inspectors  more 

attentive  to  requirements 

• Improved  detection  of  hardware 
discrepancies 

• Fewer  hardware  discrepancies 

• Better  quality  product 

• Identify  areas  needing  attention 

• More  realistic  and  explicit  process  spec 

requirements 

• Better  quality  supplier  hardware 

Manufacturing  In-Process  Audit 

• Electronic  hardware 

• Board  level  prior  to  assembly  to  package 

level 

• Reviewed  by 

• Designer 

• Packaging  engineer 

• Test  engineer 
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• Planner 

• Technician 

• Inspector 

• Reviewed  for 

• Design  against  functional 
requi  remen  ts 

• Planning  against  required  piece  parts 

• Workmanship 

• Test  data  compliance 

• Above  reviews  in  addition  to  normal 

inspection  buy-off 

Hardware  Acceptance  Audit 

• Product  review  team 

• Atlas  SLV-3D  product  reviewed  in 
factory  and  at  launch  site 

• Centaur  product  reviewed  in  factory 
and  at  launch  site 

• Atlas  F product  reviewed  in  MAB  and 
at  launch  site 


• Reviews  and  approves 

t Atlas  guidance  station  software 
operations 

• Software  design  changes,  implementa- 
tion testing 

• Software  configuration  management  and 
control 

Software  Data  Review 

• Review  members 

• Aerospace  technical  staff 

• NASA  technical  staff 

• Reviews  and  approves 

• Software  testing  requirements 

• Error  analysis  results 

• Trajectory  simulation  studies 

• Final  targeting  results 

Atlas  Guidance  Station  Review 


• All  design  disciplines  and  reliability 
represented 

• Perform  to  check  list 

• Configuration  documentation,  QARs, 
test  data,  history  jackets,  procedures, 
parameters 

• Perform  a walkdown  of  vehicle  close 
to  launch 

• Review  organization 

• Launch  crew 

• All  GDC  design  disciplines  and 
rel iabi 1 ity 

• SAI1S0/LV  and  aerospace 

• 6595th  Test  Wing  and  Aerospace 

• NASA 

Atlas  Guidance  Software  Reviews 

Trajectory  and  Guidance  Working  Group  Meetings 

• Group  members 

• SAMSO  - project  and  engineering 

• SAMTEC  - 6595th  Test  Wing  and  Range 
Operations 

• NASA  - Engineering  Staff 

• Aerospace  - Project  and  Engineering 

• Spacecraft/Upper  Stage  Contractors 

• General  Electric  - Gerts  Guidance 

• Review  and  approve  trajectory  and 

guidance  system  design  and  performance 

Software  Review  Board  (Atlas  Guidance  Station) 

• Board  members 

• SAMSO  Engineering 

• Aerospace  Engineering 

• SAMTEC  6595th  Test  Wing 

• General  Electric 


*>• 


• Review  members 

• SAMSO  - project  engineering, 
operations 

• SAMTEC  - 6595th  Test  Wing 

• Aerospace  - project,  engineering, 
operations 

• General  Electric 

• Mission-peculiar  gerts  hardware/software 
rev i ew 

• Launch  -30  day  to  launch  day  software 
exercises 

Software  Validation  Exercise 


• Review  members 

• SAMSO  engineering 

• SAMTEC  - 6595th  Test  Wing/PJG 

• Aerospace  engineering 

• Formal  mission-peculiar  software  tests  at 

Atlas  Guidance  Station 

• Validation  results  report  published 
Systems  Review 


• Review  all  launch  vehicle  systems,  con- 
figurations, solutions  to  mission 
requirements,  performance,  software, 
supporting  analyses.  This  is  accom- 
plished 30  days  before  erection  of  launch 
vehicle 

• Review  organization 

• GDC 

• Two  levels  of  independence 

• SAMSO/Aerospace 

• LV  Engineering  Office  and  Aerospace 

• LV  Operations  Office  and  Aerospace 

• LV  Mission  Assurance  Office  and 
Aerospace 


| 
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SAMTEC 


• 6595th  Test  Wing  Operations  and 
Aerospace 

• SAMTEC  Range  Operations  and  FEC 

• SA11S0  Payload  Office 

• Project  engineering  and  Aerospace 

• Others 

• Payload  contractor 

• NASA/Lewis 

• NASA/Goddard 

Stand  Up  and  Be  Counted 

• Formal  meeting  with  Director,  Launch 
Vehicle  Programs 

• One  week  before  launch 

• Key  engineering  reliability,  test  and 

factory  personnel , and  management 

• Each  individual  polled 

• Any  outstanding  worry  items 

Observations  on  Government/Contractor  Review 
Process 


• Earlier  scheduling  of  independent 
readiness  review 

• Detracts  from  concentrating  on  the 
launch 

t Too  little  time  for  corrective  action 

• Patches  instead  of  fixes 

Observations  on  Government/Contractor  Review 
Process 

• Proper  selection  of  milestones 

• Proper  scheduling  of  milestones 

Government/Contractor  Review  Process  - In  Depth 
and  Rigorous 


WORKSHOP  SUMMARY 


WORKSHOP  SUMMARY 


Industry  Mission  Assurance  Reviews  and 
Audits 


4. 


Workshop  Objectives: 

1.  To  define  appropriate  industry  management 

involvement  in  descending  review  levels 
(line  or  staff)  (to  what  depth)  (prime 
management  in  subreviews). 

2.  To  recommend  appropriate  timing  for 
industry  management  participation  in 
appropriate  review  levels. 

3.  To  explore  need  for  independence  in  the 
review  process. 

4.  To  define  role  and  volue  of  special 
audits  and  Tiger  Teams. 

5.  To  evaluate  usefulness  of  government 
review  process  and  the  timing  of  govern- 
ment reviews. 

6.  To  recommend  appropriate  industry/ 
government  mix  on  review  teams. 

7.  To  determine  if  total  government/ 
industry  review  process  is  adequate  or 
excessive  for  mission  assurance. 

Workshop  Structure: 

The  morning  session  was  devoted  to  briefings  on  the 
following  topics: 


Speakers:  Mr.  W.L.  Finch,  Lockheed 
Msls  and  Space  Co. 

Mr.  1.0.  Wolf,  Hughes  Defense 
Sys  Div. 

Mr.  W.H.  Shaw,  TRW  Defense  & 
Space  Sys  Div. 

Mr.  R.S.  Wier,  General 
Dynamics,  Convair 

Industry  Mission  Assurance  Practices  and 
procedures  were  outlined  and  discussed. 
Discussions  included  objectives,  scope, 
organization,  audit  types  and  schedules, 
checklist  and  other  audit  material. 
Significant  areas  stressed  were: 

a.  Depth  in  review  process  for  various 
management  levels,  including  the 
role  of  prime  and  submanagement 
levels. 

b.  Timing  for  management  involvement  — 
continuous  or  milestone. 

c.  Need  and  timing  for  independent 
industry/government  audits  and 
reviews . 

d.  Industry  view  of  government  review 
process. 


1 . Workshop  Overview 

Speaker:  Col.  James  A.  Smith,  SAMSG/CVI 

2.  The  SAMSO  Program  Management  Review 
Process 

Speaker:  Lt.  Col.  Gordon  Carpenter, 
SAMSO/AW 

The  background  and  evolution  of  the 
Program  Management  Review  Board  was  out- 
lined and  discussed.  Topics  included 
Pre-Contract  Design  Certification, 
Mission  Readiness,  Flight  Readiness  and 
Independent  Mission  Readiness  Reviews. 
Command  policy  statements  and  other 
material  which  set  forth  the  criteria 
and  scope  for  the  reviews  was  also 
presented. 

3.  Industry  Corporate  Visibility  for 
Mission  Success 


Speaker:  Mr.  Mark  Sasso,  RCA  Astro 
Electronics 

Corporate  policy  and  practices  were  out- 
lined and  discussed  with  respect  to 
corporate  reviews  and  audits  with 
emphasis  on  the  visibility  to  upper 
management,  level  of  management  involve- 
ment in  various  reviews,  tools,  techni- 
ques, methodologies,  and  other  manage- 
ment vehicles  used  to  provide  a proper 
balance  between  Mission  Assurance 
objectives  and  cost,  schedule  and 
performance  goals. 


5.  Summary  of  Panel  Objectives 

In  the  afternoon  session,  a panel  com- 
prised of  the  speakers  and  others  having 
expertise  in  various  related  subjects 
accepted  questions  submitted  on  cards  or 
expressed  verbally.  These  were  based  on 
issues  surfaced  by  the  morning's  presen- 
tation on  other  problems  raised  by  panel 
members. 


Lt.  Col.  G.  Carpenter 
Mr.  M.  Sasso 

Mr.  W.L.  Finch 

Mr.  I.J.  Wolf 

Mr.  W.H.  Shaw 

Mr.  R.S.  Wier 

Col . K.  Hughey 
Mr.  M.V.  Steel 
Mr.  R.E.  Ciepiela 
Mr.  W.B.  Botzong 


SAMSO/AW 
RCA  Astro 
Electronics 
Lockheed  Missiles 
& Space  Systems 
Hughes  Defense 
Systems  Division 
TRW  Defense  S 
Space  Systems  Div. 
General  Dynamics, 
Convair  Division 
SAMSO/LV 

Honeywell  Avionics 
Martin  Marietta 
RCA  Astro 
Electronics 


• Problem/Issue 

Size  of  independent  readiness  review  teams  and 
personnel  assignments  are  unsatisfactory.  This 
condition  also  applies  to  the  prime/sub  relation- 
ship. Independent  readiness  reviews  are  not  con- 
tractually applied.  Current  review  schedules 
impact  the  contractors  launch  readiness  prepara- 
tions. 
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Solution/Action 

Size  of  teams  should  be  reduced  commensurate  with 
specific  review  parameters  and  more  care  should 
be  taken  in  making  personnel  assignments.  Person- 
nel selected  should  not  be  removed  too  far  from 
main  stream  of  program.  Requirements  should  be 
projected  for  non-interference  with  launch  readi- 
ness activity.  (This  problem  may  already  have 
been  partially  resolved  through  SAMSO  pamphlet 
800-11  which  has  not  yet  been  communicated  to  all 
contractors. ) 

Responsible  Agency  - SAMSO/Contractors 

• Problem/lssue 

Independent  review  process  timing  is  not 
optimized. 

Solution/Action 

Examine  the  possibility  of  establishing  an  indepen- 
dent design  review  as  soon  as  possible  in  the  CDR/ 
PDR  time  frame.  This  area  should  be  emphasized 
and  should  not  interfere  with  additional  activity 
as  described  in  "running  cell  acceptance," 
"progressive  acceptance,"  and  "pedigree  packages." 

Responsible  Agency  - SAMSO/Contractor 

• Problem/lssue 

Launch  readiness  review  should  be  treated  as  a 
process  rather  than  an  event. 

Solution/Action 

Use  a system  which  is  equivalent  to  LMSC's  "run- 
ning sell,"  the  Hughes'  "progressive  acceptance" 
concept,  and  Martin  Marietta's  "pedigree  package" 
concepts. 

Responsible  Agency  - SAMSO/Contractor 

• Problem/lssue 

Flexible  agenda  - periodic  management  reviews, 
i.e.,  monthly/bi-monthly,  often  have  rigorous 
contractually  dictated  agenda. 

Solution/Action 


contractor  and  government,  can  understand  issues 
and  objectives.  Team  meetings  should  be  held  to 
create  criteria,  checklists,  types  of  questions  to 
ask,  and  to  prepare  reviewers  with  techniques  to 
be  used. 

Responsible  Agency  - Contractors 

• Problem  Issue 

Special  review  debriefings:  An  adequate  debriefing 
to  those  reviewed  should  be  accomp] ished. 

Solution/Action 

Review  groups  should  always  document  conclusions. 
Debrief  responsible  personnel,  and  be  sure  action 
item  requirements  are  understood.  Conclusions 
should  be  documented.  Action  items  should  be 
unambiguously  assigned. 

Responsible  Agency  - Contractors 

• Problem/lssue 

Transfer  of  technical  review  findings  is  unsatis- 
factory. This  applies  to  both  contractor  and 
government  special  and  independent  readiness 
reviews . 

Action/Solution 

Cross  program  knowledge  should  be  provided  in  non- 
proprietary areas  between  programs  by  both  the 
government  and  the  contractor. 

Responsible  Agency  - SAMSO/Contractor 

• Problem/lssue 

Reviews  are  always  added  — never  deleted 
Solution/Action 

Review  SAMSO  Reg  800-3  for  possible  elimination  of 
reviews  due  to  redundancy  or  possibility  of  com- 
bining reviews.  Assure  those  contractually 
specified  are  applicable  and  tailored  to  that 
contract. 

Responsible  Agency  - SAMSO 


Periodic  management  reviews  should  have  a flexible 
agenda  depending  on  status  of  program.  Customer 
SPO  director  and  contractor  project  manager  have 
a pre-ineeting  to  establish  agenda  and  designate 
participants. 

Responsible  Agency-  SAMSO 

• Problem/lssue 

Inadequate  review  preparation 

Solution/Action 

Review  effectiveness  can  be  improved  by  pre- 
planning with  the  team  and  preparing  reviewers 
with  suggested  techniques.  Development  of  check- 
lists may  be  part  of  this  process.  Reviews  should 
be  pre-scheduled  and  event-oriented.  Team  should 
be  picked  with  good  lead  time.  The  review  plan 
should  be  documented  so  that  all  team  members,  both 
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WORKSHOP  D 

TECHNIQUES  FOR  MANAGEMENT  AND  CONTROL  OF  THE  DEVELOPMENT 
AND  APPLICATION  OF  SYSTEMS  COMPUTER  SOFTWARE 


Workshop  Coordinator 

Mr.  Dan  Browne,  SAMSO/AQ 


Dr.  Eldred  C.  Nelson,  Director 
Technology  Planning  and  Research 
Systems  Engineering  and  Integration  Division 


Co-Chairmen 

Mr.  Robert  E.  Berri 
Member  of  the  Technical  Staff 
The  Aerospace  Corporetion 


AGENDA 

Wednesday,  April  26 

0630-0645 

Introduction 

Mr.  Robert  E.  Berri,  Aerospace 

0845-0915 

Overview 

Dr.  Barry  W.  Boehm,  TRW 

0915-0645 

SAMSO  Software  Environment 

Mr.  Robert  E.  Berri,  Aerospace 

0945-1000 

Break 

1000-1030 

FMEA  for  Software 

"Mr.  Myron  Lipow,  TRW 

1030-1100 

Software  Design  Standards 

Mr.  James  P.  Chilton,  MDAC 

1100-1130 

Software  Quality  Assurance 

Mr.  Harvey  1.  Gold,  SDC 

1130-1300 

Lunch 

1300 

Open  Discussion 

INTRODUCTION 


Mr.  Robert  E.  Berri 
Technical  Staff 
The  Aerospace  Corporation 


Software  Workshop  Panelists 


Len  Anding 

General  Dynamics 

QA 

Dan  Browne 

SAMSO/AW 

STE 

Tom  Burns 

Hughes 

STD 

Clell  Gladson 

Contract 

Administration 

MGMT 

Shirley  McCarty 

Aerospace 

STD 

Leon  Rue 

Rockwel 1 

FMEA 

A1  Schal lenmul ler 

Martin  Marietta 

STD 

Roger  Shol ten 

Boeing 

QA 

Barry  Zilin 

SAMSO/ PM 

Procure 

Summary  Goals 

• Define  present  SAMSO  counterproductive 
practices  or  philosophies 

• Make  cost  effective  recommendations 

• Recommend  SAMSO  operations  standards 

• Establish  if  there  is  a need  for  on-going 
sessions 

• Topics 

• Criticality 

• Priorities  of  the  above 


SOME  KEY  SOFTWARE  ISSUES 

Dr.  Barry  W.  Boehm, 
Director,  Software  Res  and  Tech 
TRW 


TECHNIQUES  EXIST  FOR  CATCHING  SOFTWARE  ERRORS  EARLY 


General  software  mission  assurance 
concerns 

Some  specific  SAMSO  concerns 

• Present 

• Future 

Conclusions:  an  analogy 


MOST  SOFTWARE  ERRORS  ARE  MADE  IN  DESIGN  PHASE 


ttMCfNTOT  VALIDATION  IISI  tHNOAs  COVtMO 


SUBSYSTEM 
ACCEPTANCE  TEST 
AND  SYSTEM 
INTEGRATION  ERRORS 


GETTING  ERRORS  OUT  EARLY  RESUl  TS 


ACQUIRE  MINTS  VALIDATION 
DESIGN  VERIFICATION 
STANDARDS 
WALK  THAOUONS 

RCVlfWS 


DESIGN  - 64% 


t A*<  It  » TAW 

r«Oj*Ci 
I it>i  aiiNCl 


* TRW  CCIP-85  DATA:  220  ERROR  TYPES 


REQUIREMENTS  ERRORS  MUST  BE  CAUGHT  EARLY 


Relative 
cost  to 
lit  errot 


Some  Specific  SAMSO  Concerns:  Present 


SAFEGUARD 


Mission  software  problem  causes  and 
countermeasures 

• Interface  problems 

• Data  load  problems 

• Control  problems:  who's  in  charge? 

• CM  and  QA  problems 

• Off-nominal  testing 

e Simulation  and  support  software 


Require 

ments 


Phase  in  which  error  delected 


How  to  determine  appropriate  level  of 
firmware  - life-cycle  maintenance 
implications 

How  to  plan  and  carry  out  microprogram 
testing 

How  to  oet  better  microprogram  standards 
and  "HOL's" 


SOFTWARE  MANAGEMENT  WHERE  ARE  tVf  GOING? 


LIFE -CYCLE  PROBLEMS 

• MCftO  PROCESSOR.  FlKMHMf  AC  XAS4TION 

• CONCURRENT  SOf  TWARE  DESIGN 

• TAILORED  SUPPORT  SOFTWARE 

• MAINTENANCE  MANAGEMENT 


DISTRIBUTED  DATA  PROCESSING  HAS 
FUNDAMENTAL  RELIABILITY  PROBLEMS 


RESOURCE 

POOL 


DEADLOCK 


OBTAIN  A 
THEN 

RELEASE  B 


MISSILE  AND  SOFTWARE  ERROR  DEFENSE  MEASURES  I 


RACE 


STORE 


READ 


UPDATE  Y 


Distributed  Data  Processing  Reliability:  Ke 
Issues 


How  to  detect,  avoid  deadlock  and  race 
problems 

How  to  instrument  concurrent  processors 

How  to  write  DDP  Specs,  Acceptance 
criteria 

How  to  revalidate  a modified  DDP  System 
How  to  specify  appropriate  standards 
• E.G.,  DoD-1 


MISSILE  AND  SOFTWARE  ERROR  DEFENSE  MEASURES  II 


Microprogram  Reliability:  Key  Issues 


How  to  detect  and  avoid  dverse  side 
conditions 

How  to  instrument  microprograms 

How  to  assure  fidelity  of  simulators  + 
emulators 

How  to  detect  and  avoid  low-level  race 
problems 


XN 

X 

CPU, 

4 — — - 

Y 

+z — 
VN*1 

cpu2 

OUCCTWt 

MISSIVE  DEFENSE 

SOFTWARE  ERROR  DEFENSE 

REDUCE  PLAUWC„ 

COUNTERFORCE 

LANGUAGES.  STANDARDS.  ME  THOOS 

FOR 

SALT 

REQUIREMENTS.  DESIGN.  PROGRAMMING 

INCREASE  EARLY  PDtuo 

SPACE  SENSORS 

ROTS.  DESIGN  COOE  ANALYZERS 

1 f DISCRIMINATION 

RADARS 

WALK  THRUS.  SIMULATION 

* ^INTERCEPT 

INTERCEPTORS 

INCREASE  EARLY  IANO 

TERMINAL  HOMING 

RE  VALIDATION  OF  FIXES 

TERMINAL)  PK|LL 

KILL  MECHANISMS 

RIGOROUS  CM.  QA 

Some  Observations  About  Software  Error  Defense 


• Overemphasis  on  terminal  defense 

(testing) 

• Need  more  effort  on: 

• Reducing  error  launch  rate 

• Increasing  early  detection  — — 
elimination  capabilities 

• Increasing  probability  of  surviving 
effects  of  errors 

• Need  more  effort  in  threat  analysis 

• Comparable  to  intelligence  budget 

• Study  implications  of  new  technology 
(micros,  firmware,  DDP,  MCE,  DoD-1) 

• Need  a focal  point  of  responsibility 


SAMSO  SOFTWARE  ENVIRONMENT 

Mr.  Robert  E.  Berri 
Technical  Staff 
The  Aerospace  Corporation 


Issues 

• Should  SAMSO  standardize/ control  launch 
environment? 

• Software 

t Special  test  eauipmont 

• Can  range  (SAMTEC/AFSCF)  interfaces  be 
improved? 

• Should  computer  resources  working  groups 
be  utilized? 

• Membership 

• Functions 

• How  extensively  should  test  plans/ 
procedures  be  reviewed? 

• Is  independent  V&V  cost  effective? 

• Should  internal  (e.g.,  analysis)  soft- 
ware be  more  portable? 


FAILURE  MODES  AND  EFFECTS  ANALYSIS 
FOR  SOFTWARE 


Myron  Lipow 
Sr.  Staff  Engineer 
TRW 

M.  Lipow 
A.  Frimtzis 
D.  Reifer 

Software  Failure  Modes  and  Effects  Analysis 
Topics: 

• The  purposes  of  FMEA  - definitions 

• Prevention  of  failure  modes 

• Compensating  features  for  single  point 

failure  modes 

• Fault  tolerance  features 
The  Purposes  of  FMEA: 

• To  identify,  rank,  and  compensate  for 
failure  modes  of  critical  functions 

• As  a tool  for  design  evaluation  and  test 
planning 

• To  provide  input  to  reliability,  main- 

tainability, and  safety  analysis 


Classification  of  Failure  Modes 

• Hardware- induced  hardware  failure  modes 

and  effects  - SAMSO  77-2 

• Software- induced  hardware  failure  modes 

and  effects  - SAMSO  77-2 

• Logic  error  results  in  arming  fuze  at 
wrong  altitude  or  not  at  all  - SAMSO 
77-2 

• Hardware-induced  software  failure  modes 

and  effects 

i Transient  spike  results  in  abnormal 
input  data 

• Software-induced  software  failure  modes 

And  effects 

• Poor  algorithm  approximation  results 
in  out  of  tolerance  computation 


Software  FMEA 

• Definitions 

• Software  - computer  programs  + data 

• Failure  mode  - manner  in  which  fail 
ure  occurs 

t Examples: 

• Program  aborts 

• Program  degrades 

• Program  loops  indefinitely 


• Failure  effect  - consequence(s) 
of  the  failure  mode 

• Single  point  failure  - at  any  level, 
the  failure  of  which  would  result  in 
irreversible  degradation  of  mission 
performance,  or  prevent  achievement 
of  a mission  objective 

Purpose  and  Application  of  the  FMEA 
( SAMSO-STD-77-2 ) : 

1.  To  identify  all  failure  modes-mission 
effects-corrective/compensating  action 

2.  To  identify  single  point  failure  modes 

3.  To  identify  critical  functions  for 
redundancy 

4.  To  identify  compensating  features 

5.  To  identify  untestable  redundant  areas 

6.  To  aid  in  identifying  untestable 
functions 

7.  To  rank  most  serious  failure  modes 

8.  To  establish  a critical  items  list 

9.  To  form  an  input  to  reliability  modeling 
' and  predictions 

10.  Iterative  design  tool 

11.  Evaluation  design  tool 

12  Diagnostic  tool 

13.  Test  planning  criterion 

14.  Determining  operational  constraints 

15.  Aid  in  maintainability,  safety,  hazards 
analysis 

16.  Identify  problem  areas  to  be  avoided  in 
manufacturing. 

Potential  Failure  Modes  Introduced  in  Design  and 
Coding 

• Requirements  incomplete  or  ambiguous 
leading  to  deficient  design 

• Coding  practices  (lack  of  standards, 
inspection)  leading  to  difficult-to- 
detect  faults 

• Non-robust  algorithms 

• Fail  to  converge 

• Converge  at  sub-optimal  points 

t Deadlock  - in  distributed  processors  - 

problem  because  of  inadequate  memory 
margin 
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SOFTWARE  FAULT  TOLERANCE 


• SELC-CHECKING  SOFTWARE 

• REDUNDANT  SOFTWARE 


EXAMPLE:  GAC  SYSTEM 


m 

18K 

33K 

BASIC 

ADO: 

ADD: 

P90F.PA1 

MSTRtmTATlOH 

REDUNDANCY 

SINGLE  POINT 
FAILURE  "ODES 


HEALTH  AND  STATUS 
CHECKING  IN  REAL  TINE 

INFORMED  NANUAl 
RECOVERY  POSSIBLE 


REDUNDANT  SOFTWARE 
WITH  SWITCHING  LOGIC 

NAY  HOT  PROVIDE  ALL 
FUNCTIONS 


• AEROSPACE  STUDY  OF  FLIGHT  CONTROL  SOFTWARE 

• 30T  WORE  STORAGE 

• ION  PORE  EXECUTION  [INF 


• Unauthorized  or  inadvertent 
launcn 

• Premature  launch 


NSCCA 


• Unauthorized  display  of 
classified  information 


• Faulty  launch 

• Invalid  verification  of  codes 


• Satisfaction  of  program 
specifications 


PATE 


• Extremes  testing 

• Program  does  not  degrade  sub- 
system capability 


Compensating  Features  for  Single  Point  Failure 
Modes 

To  reduce  the  probabilities  of  single  point  fail- 
ure modes  to  acceptable  values. 

How?  Techniques  and  tools  to  prevent/detect 
errors/faults. 

• Design 

• Top-down  - stepwise  refinement 

• Requirements  verification/ 
traceability  systems 

• Program  design  language 

• Design  walkthroughs 

• Programming 

• Standards  - compliance  tool 

• Structured  programming  - e.g., 

S-Fortran 

• Code  walkthroughs 

• Static  analyses 

• Set-use  checker 

• Units  consistency  checker 

• Phantom  path  analysis  - complexity 
assessment  and  reduction 


DISTRIBUTED  DATA  PROCESSING  HAS 
FUNDAMENTAL  RELIABILITY  PROBLEMS 


DEADLOCK 


OBTAIN  B 
THEN 

RELEASE  A 


RACE 


READ 
XN-  VN 


RESOURCE 

POOL 


STORE 


OBTAIN  A 
THEN 

RELEASE  B 


UPDATE  Y 


Research  Directions 


Near-Term 


• Modify  MIL-STD-1543  and  SAMSO  Standard 
77-2  to  include  software  FMEA 

• Analyze  RADC/IS  software  reliability  data 
base  to  determine  failure  modes 

• Identify  how  software  FMEA  can  be  used 
as  part  of  system  reliability  analysis 


• Testing 

• Frequency  of  loop  usage 

• Aids  to  identify  test  cases 

• Interpretive  computer  simulation- 
timing 

• Nuclear  safety  cross-check  analysis 
(NSCCA) 

Preventing  Failure  Modes  for  the  Minuteman  Missile 
System  Software  (Independent  V&V  Contractor! 


Long-Term 

• Identify  design  and  verification  and 
validation  practices  needed  to  reduce 
failure  modes 

• Identify  certification  procedures  that 
can  be  used  and  related  facility  con- 
cepts (simulators,  emulators,  etc.) 

• Develop  a system  reliability  model  con 
cept  that  includes  the  incremental  con 
tribution  of  software 


• Nuclear  safety  cross-check  analysis 
(NSCCA) 

• Performance  and  technical  evaluation 
(PATE) 
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SOFTWARE  DESIGN  STANDARDS 


SOFTWARE  DEVELOPMENT  APPROACH 


James  P.  Chilton 

Director,  Data  Processing  Sys  Tech  Prgms 
McDonnell  Douglas  Astronautics  Company 


Air  Force  Software  Concerns 


Significant  and  growing  problem 
• Lack  of  controls 


SOFTWARE 

• WILD  REQUIREMENTS 

ENTHUSIASM 


POTENTIAL  RESULTS 

• LATE  SYSTEM  DELIVER^ 

• COST  OVERRUNS 

• OLGRAOEO  SYSTEM  PERFORMANCE 

• UNRELIABLE  PRODUCT 


PRELIMINARY 

• DISLLUSIONMENT  DESIGN 


Black  art 


• TOTAL  CONFUSION 


Difficult  to  verify 
Training  needed 


CODE  AND 

• SEARCH  FOR  THE  GUILTY  OEBUG 


Government 


SAN 

TEST  AND 
INTEGRATION 


t Industry 

Should  have  same  disciplines  that  are 


SYSTEM 

• PROMOTION  OF  NONPARTICIPANTS  TEST 


applied  to  hardware 


Applies  to  both  deliverable  and  non- 
deliverable 


Key  Elements  of  Software  Development  Approach 


What  Can  We  Do  To  Assure  Successful  Software? 


• Management  actions 

• Development  approach 


Quality  assurance 


Management  Actions 


Software  development  team 
Software  requirements 
Setting  software  design  baseline 
Plans,  procedures,  standards 
Top  down  design  and  implementation 
Design  reviews  and  visibility 
Documentation  and  traceability 
Configuration  management 
Testing 


Recognize  that  software  is  a product  and 
can  be  manufactured 


Establish  disciplined  development 
approach 

Establish  management  controls 


SOFTWARE  DEVELOPMENT  TEAM 


Plan  quality  assurance  from  the  start 


SOFTWARE  DEVELOPMENT  APPROACH 


DEVtlOPMENT  PHASES 


Software  Requirements 


Represent  the  system  design  solution 

Can  be  determined  only  as  part  of  the 
engineering  of  the  system 

Are  the  basis  for  defining  the  desired 
Implementation  to  the  software  developer 

Must  be  testable 


PRELIMINARY  DESIGN 
THE  DOWNWARD  ALLOCATION  OF  REQUIREMENTS 
AND  DESIGN  TO  A SOFTWARE  STRUCTURE 


Complete  Preliminary  Design  Forces  Establishment 
of  a Sound  Development  Baseline 


IMPLEMENT  THE  SOFTWARE  FROM  THE  TOP  DOWN 


• Provides  solid' software  design  feedback 
to  customer/system  'designers 

t Provides  software  structure  for  managing 
detail  design 

• Establishes  the  capability  of  the  process 
structure  to  accommodate  change 

• Determines  port-to-port  responses  in  a 
realistic  environment  - under  load 

• Establishes  adequacy  of  interface 
design 

• Establishes  that  the  operating  system 
and  development  support  software  will 
support  the  applications  programs 


V 


Establishing  a Design  Baseline  is  More  Than  Just 
Requirements  and  Preliminary  Design 


DESIGN  REVIEWS 


• S/W  development  plan 


DEVELOP  SOFTWARE  INCREMENTALLY  TO  MINIMIZE  RISK 


• Configuration  management  plan 

• Product  assurance  plan 

• Management  procedures 

• Project  reporting  and  management 
t Security 

• Configuration  management 

• Software  standards  and  procedures 

• Naming  conventions 

• Programming  standards 

• Documentation  conventions 

• Unit  test  criteria 

• Software  quality  manual 

t Software  validation  test  plan 
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DOCUMENTATION 


• STANDARDS 
PROCEDURES 
MANUALS 


Configuration  Management 

• All  computer  programs  progress  through 
an  ordered  set  of  baselines  consistent 
with  program  schedules 

• Approved  software  requirements  and  inter- 
face specifications  are  the  basic  design 
requirements 

• Software  baselines  include  requirements, 
design,  code  and  product  representing 
progressively  more  stringent  levels  of 
control 

• Changes  must  be  approved  by  the  appro- 
priate change  control  board 


LEVEL  OF  PRODUCT  CONTROL 
VARIES  WITH  DEVELOPMENT  PHASE 


OCL 

REQUIREMENTS 

PREl  1 
OES 

DET 

OES 

CODE 

C/0 

PROCESS 

INTEG 

PROCESS 

VALID. 

SYS 

l/T 

PROGRAM  MANAGER 

X 

X 

SOFTWARE  MANAGER 

X 

X 

PROCESS  MANAGER 

* 

X 

X 

X 

TASK  MANAGER 

X 

X 

ROUTINE  MANAGER 
ROUTINE  MANAGER 

1 

TASK  MANAGER 

X 

X 

ROUTINE  MANAGER 
ROUTINE  MANAGER 

TASK  MANAGER 

X 

X 

Qual i ty  Assurance 

• Separate  function 

a Participate  in  review  of  documentation, 

standards  and  procedures 

• Utilize  automated  tools  to  verify  com- 
pliance with  standards  and  procedures 

• CCB  participant 

• Verify  test  criteria  compliance 

• Verify  content  of  delivered  product 

Conclusions 

• A disciplined  approach 

• Enforced  by  management 

• With  appropriate  visibility  and  controls 

• Results  in  successful  software  which 

• Performs  its  required  functions 

• Under  prescribed  conditions 

• On  cost 

• On  schedule 

Discussion  Points 

• Value  of  a disciplined  approach 

• Value  of  government  imposed  regulations 
to  software  quality 

• Cost  of  having  standards  versus 
guidel ines 

• Cost  of  visibility  and  is  it  worth  it 

• When  and  where  is  quality  assurance  most 
effective  in  achieving  quality  software 
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SOFTWARE  QUALITY  ASSURANCE 

Mr.  Harvey  I.  Gold 
Department  Manager 
System  Development  Corporation 


SOFTWARE  QUALITY  ASSURANCE 


DEFINITIONS 

QUALITY  ASSURANCE 


SOFTWARE  QUALITY  ASSURANCE 


IMPLEMENTING  THE  CONTRACTOR’S 
QUALITY  ASSURANCE  PROGRAM 

• MILS  62779 

- REQUIRES  A SOFTWARE  QA  PROGRAM  TO  ASSURE  THAT  ALL  SOFTWARE 
PROOUCTS  MEET  ALL  CONTRACTUAL  REQUIREMENTS 

• RFP  REQUIRES  SOFTWARE  QA 

- SHOULD  BEGIN  IN  VALIDATION  PHASE 

- QA  PLAN  PART  OF  THE  CP  DP 

• CONTRACTOR  S QA  PROGRAM  ADDRESSES: 

- WORK  TASKING  » AUTHORIZATION  PROCEDURES 

- CONFIGURATION  MANAGEMENT 

- TESTING  ft  TESTABILITY 

- CORRECTIVE  ACTION 

- LIBRARY  CONTROLS 

- COMPUTER  PROGRAM  DESIGN 

- SOFTWARE  DOCUMENTATION 

- REVIEWS  * AUDITS 

- TOOLS.  TECHNIQUES  t METHODOLOGIES 

- SUBCONTRACTOR  CONTROL 


PLANNING  THE  CONTRACTOR’S 
QUALITY  ASSURANCE  PROGRAM 


PERSONNEL  QUALIFICATIONS 


HELP  WANTED 

POSITION  - SOFTWARE  QUALITY  ASSURANCE  STAFF 
QUALIFICATIONS  - 

EXPERIENCE  IN  FOLLOWING  AREAS 

• MILITARY  OPE  RAT  IONS  KNOWLEDGE 

• SOFTWARE  AOUISITION  MANAGEMENT 

• CONFIGURATION  MANAGEMENT 

• DATA  MANAGEMENT 

• RELATED  MILITARY  SPECIFICATIONS  A STANDARDS 

• TECHNICAL  EXPERIENCE  IN  SOFTWARE  DEVELOPMENT 

• CON  TRAC  TING /SUBCONTRACTING 

• TEST  MANAGEMENT 

• SOFTWARE  ENGINEERING  MANAGEMENT 
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TOOL# 


METHODOLOGIES 


• Some  important  organizational  issues: 

• Is  QA  a line  or  a staff  function? 

• Who  does  QA  report  to? 

• What  authority  (or  power)  does  QA 
have? 

• Adversary  or  cooperative  role? 


RLLAIieflSMP -ID  JUltltO 


GOVE  TOM 
(CUSTOMER) 

DEVELOPMENT 

CO’lTRACTOP 

indepe:pent 

COfimTOR 

SETD 

V 

DEVELOP- 
MENT TEST 

V 

INDEPENDENT 

TEST 

V 

V 

QA 

V 

V 

• QA  REVIEWS  ARID  EVALUATES  THE  TEST  PTTOCESS  ( PRODUCTS 

• QA  COES  tlQI  DC  TESTING 


Discrepancy  Reporting/Correction 

• Should  QA: 

• Report  problems? 

• Maintain  problem  status? 

• Correct  problems? 

• QA  should: 

• Evaluate  problem  handling  plans 

• Evaluate  the  process 

• Produce  QA  evaluation  reports 

• Maintain  QA  records 


WORKSHOP  SUMMARY 


Issues  Raised  and  Discussed 

Issues  raised  by  the  speakers  and  discussed  by  the 
panels  are  summarized  below.  Because  some  of  the 
issues  were  common  to  several  of  the  topic  areas, 
the  issues  are  presented  as  software  mission 
assurance  issues  rather  than  associating  them  with 
specific  topic  areas. 


1.  Software  errors  affecting  mission  assur- 
ance frequently  originate  in  require- 
ments and  design  phases;  therefore 
software  errors  should  be  addressed  in 
these  phases.  Use  of  new  techniques  and 
tools  supporting  the  identification  and 
analysis  of  these  errors  - e.g.,  require- 
ments validation,  design  veri fication, 
standards,  walk  throughs,  and  reviews  - 
should  be  encouraged  by  SAMSO. 

2.  Testing  limited  to  nominal  specification 
values  tends  to  be  inadequate.  Test 
strategy  should  include  consideration  of 
extreme  and  singular  points. 

3.  SAMSO  needs  an  organization  like  the 
AFSC/XRF  Computer  Resources  Committee  - 
to  initiate  SAMSO  software  standards, 
transfer  lessons  learned,  and  review 
computer  resource  acquisition. 

4.  SPO's  should  be  better  informed  on  how 
software  can  affect  mission  assurance. 

Data  on  software  failure  history  in 
space  and  missile  systems  is  needed. 
Analysis  of  the  software  data  base 
collected  by  RADC  may  help. 

5.  Should  project  management  plans  be  con- 
tract compliance  documents?  Although 
opinions  were  divergent  on  this  issue, 
it  was  agreed  that  such  plans  are  valu- 
able management  aids  to  both  SAMSO  and 
the  contractor.  Their  use  should  be 
encouraged. 

6.  Hazard  analysis  using  FMEA  techniques 
and  simulations  should  contribute  to 
mission  assurance.  SAMSO  Standard  77-2 
should  be  modifieo  to  include  software. 

7.  A limited  version  of  configuration  man- 
agement and  quality  assurance  should  be 
imposed  on  software  test  tools. 

8.  Upper  management  (both  SAMSO  and 
contractors)  need  to  better  understand 
the  software  development  process  and  its 
role  in  system  development.  A manage- 
ment education  program  on  software  needs 
to  be  developed. 

9.  SPO  planning  generally  fails  to  recognize 
logistic  support  and  operational  use  prob- 
lems, particularly  as  they  affect  soft- 
ware. Such  considerations  should  be 
addressed  in  the  system  concept  phase. 


10.  The  software  design  phase  can  benefit 
from  effective  management  and  should 
include  test  planning  and  consideration 
of  operational  use  procedures. 

11.  SAMSO  review  of  contracts  should  be 
strengthened  to  assure  (a)  adequacy  of 
software  requirements  such  that  they  are 
complete,  clear,  unambiguous,  and  under- 
stood by  both  parties,  (b)  invocation  of 
recent  policy  on  configuration  management, 
modern  programming  practices,  and  design 
reviews,  and  (c)  adequate  use  of  manage- 
ment plans. 

12.  Software  quality  assurance  can  contribute 
to  mission  assurance.  SAMSO  needs  to 
define  what  is  required  of  the  contractor. 

13.  M1L-S-52779,  when  tailored  by  SAMSO P 74-2, 
is  a good  frame  of  reference  for  quality 
assurance,  but  is  needs  to  be  modified  to 
deal  more  specifically  with  software 
needs. 


ISSUES  RAISED  BUT  NOT  DISCUSSED 

Owing  to  lack  of  time  and  to  the  focus  of  interest 
of  the  Workshop  attendees,  some  of  the  issues 
raised  by  the  speakers  were  not  discussed,  to  any 
degree,  by  the  panels  and  the  audience.  These 
issues  are  summarized  here: 

1.  The  AFPRO  role  in  monitoring  software 
development  e.g.,  with  respect  to 
design  standards,  intermediate  testing, 
(PQT)  monitoring,  and  configuration 
management  - could  be  strengthened. 

2.  Distributed  processing,  particularly  in 
systems  involving  microprocessors,  intro- 
duces new  problems  affecting  mission 
assurance  e.g.,  race  conditions  and 
deadlock. 

3.  SAMSO  launch  control  areas  are  essenti- 
ally program  peculiar.  Could  standardi- 
zation of  this  environment  be 
accomplished,  benefiting  both  hardware 
and  software  development? 

4.  Interfaces  between  SPO's  SAMTEC,  and 
AFSCF  appear  to  not  be  well  defined.  Is 
an  improvement  necessary? 

5.  Since  software  in  embedded  computer  sys- 
tems generally  has  not  been  tested  as  a 
configuration  item,  should  more  extensive 
software  testing  be  required? 

6.  Is  independent  verification  and  valida- 
tion effective  for  SAMSO  programs? 

7.  SAMSO  contractors  utilize  extensive 
Internally  developed  development  support 
software.  Since  some  of  this  is  believed 
to  be  redundant,  should  SAMSO  require 
delivery  and/or  portability  of  it? 
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8.  Because  major  change  orders  (ECPs)  can  be 
processed  at  SAMSO  without  adequate  staff 
review,  their  effect  on  mission  assurance 
may  not  always  be  considered.  Is  better 
control  needed? 

9.  A system  reliability  model  including  the 
software  contribution  is  needed. 

10.  Better  methods  of  verification,  valida- 
tion, and  certification  are  needed  to 
reduce  failure  modes. 

11.  Contracts  tend  to  overconstrain 
contractors. 

12.  Are  there  cost  advantages  in  not  imposing 
software  design  standards? 

13.  What  should  be  the  relative  role  of  soft- 
ware quality  assurance  and  software 
testing? 

Principal  software  development  problems  affecting 
mission  assurance  are  management  problems 

Reviews  do  not  adequately  address  issues  affecting 
mission  assurance 

Cost  Saving  Actions  at  Procurement  Time 

• On: 

• Documentation 

• Test 

• QA  Functions 

• Frequently  increase  mission  assurance 
problems 

Software  Workshop  Recommendations 

SAMSO  should  establish  counterpart  of  DSARC  reviews 

• To  address  software  issues  affecting 
mission  assurance 

SAMSO  should  strengthen  role  and  resources  of 

• Focal  point  for  computer  resources  . 
Software  life  cycle  cost  and  logistic  support 

• Should  be  considered  at  concept  stage 

• Need  software  life  cycle  cost  model 

Follow-on  session  desireable 
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WORKSHOP  E 

SUBCONTRACTOR  INTERFACE  & CONTROL  TO  ACHIEVE 
MISSION  ASSURANCE 

Co-Chairman 

Lt  Col.  Richard  E.  Tracey  Mr.  Howard  C.  Ringoen 

Chief  of  Reliability  & QA  Director  Material 

ICBM  Program  Office  Boeing  Aerospace  Co. 


AGENDA 

Wednesday,  April  26, 

0630-0845 

Introductions,  Objectives,  Agenda 

Mr.  Howard  C.  Ringoen,  Boeing 

Mr.  Richard  D.K.  Goo,  SAMSO/MNCP 

0845  0915 

Procurement  Trends  in  Small  and 
Minority  Business 

Mr.  Eutimio  B.  Romero,  Boeing  Co. 

0915-0945 

Controlling  the  Independent,  Profit 
Motivated  Subcontractor 

Mr.  Peter  H.  Fowler,  TRW  Systems 

0945-1015 

Break 

1015-1045 

Critical  Elements  in  Supplier  Control 

Mr.  Marion  Smith,  Honeywell,  Inc. 

1945-1115 

Comments 

Attendees 

1115-1315 

Assignments  for  Afternoon  Panel 
Discussions 

1315-1600 

Panel  Discussions 

1600 

Panel  Conclusions  and 

Recomendations 

Discussion  Moderators: 

Mr.  Andy  Myers 

Mr.  Pete  Fowler 

Mr.  Marion  Smith 

Martin  Marietta  Aerospace 

TRW  Systems 

Honeywell  Inc. 

Overall  Objective: 

To  provide  a forum  for  open  and  frank  dialogue  between  the  Government  and  Industry  on 
specific  subcontractor  problems  and  concerns,  the  correction  or  improvement  of  which  will 
provide  greater  mission  assurance. 

Specific  Goals: 

1.  To  define  present  SAMSO  practices  or  philosophies  that  are  counterproductive 

2.  To  develop  recommendations  that  will  provide  cost  effective  mission  assurance 
a To  define  recommended  standards  of  operation 

4.  To  define  requirements  for  on-going  studies 


WORKSHOP  E (continued) 

Specific  Areas  to  be  Discussed  by  each  Panel  where  Improvements  would  Enhance 
Mission  Assurance 

1.  Flow  down  of  motivation  (fees/disengagement) 

2.  Flow  down  of  critical  parameters,  considerations  and  design  requirements 

3.  Selecting  the  subcontractor/supplier 

4.  Pre-award,  post-award  and  facility  capability  surveys  (audits) 

5.  Optimizing  source  acceptance 

6.  Monitored  fabrication  lines  (process  controls)/coordinated  procurements 

7.  Visibility  and  trust  in  subcontractor/supplier  operations/relations  (types  of 
contracts) 

8.  Small  lot,  high-reliability  buying  options 

9.  Material  review  boards,  preliminary  reviews  and  corrective  action 

10.  Other 


CURRENT  SMALL/MINORITY  BUSINESS  TRENDS 
IN  PROCUREMENT 

Mr.  Eutimio  B.  Romero 
Boeing  Company 


• Established  smal 1/minori ty  business 
programs 

• Industry  approach 

• Environment 

• Concerns 

• Conclusions 
Small  Business  Definition 

A small  business  concern  is  independently  owned 
and  operated,  is  not  dominant  in  the  field  of 
operations  in  which  it  is  bidding,  and  with  its 
affiliates,  can  further  qualify  under  the  criteria 
set  forth  in  ASPR  1-701  or  NASA  PR  1-701.  "Concern" 
means  any  business  entity  organized  for  profit  with 
a place  of  business  in  the  United  States,  its  pos- 
sessions, Puerto  Rico,  or  the  Trust  Territory  of 
the  Pacific  Islands,  including  but  not  limited  to 
an  individual  partnership,  corporation,  joint  ven- 
ture, association,  or  cooperative. 

Minority  Firm  Definition 

The  term  "minority  business  enterprise"  means  a 
business,  at  least  50  percent  of  which  is  owned  by 
minority  group  members  or,  in  case  of  publicly 
owned  businesses,  at  least  51  percent  of  the  stock 
of  which  is  owned  by  minority  group  members.  For 
the  purposes  of  this  definition,  minority  group 
members  are  American  Blacks,  Spanish-speaking 
American  persons,  American  Orientals,  American 
Indians,  American  Eskimos,  and  Americal  Aleuts. 

Contractual  Obligations  (ASPR  1-700) 


• Cooperate  with  contracting  officer  in 

studies  and  surveys 

• Submit  quarterly  reports  to  Government 

• Contract  may  be  terminated  for  failure 

to  comply 

• Maintain  records 

• Size  of  suppl iers 

• Procedures  for  complying  with  this 
clause 

a Purchase  of  contracts  SI  OK  and  up, 
whether 

• Awarded  to  large  or  small 

business 

• Number  of  small  businesses 
sol ici ted 

• Reasons  for  not  soliciting 

small  businesses 

• Reasons  small  businesses 

failed  to  receive  award  when 
solicited 

Corporate  Policy 

• It  is  Boeing  policy  not  only  to  comply 
with  its  contractual  obligation,  but 
also  to  place  a fair  orooortion  of  its 
total  purchases  for  materials,  supplies, 
and  services  with  small  businesses, 
minority-owned  businesses,  and  labor 
surplus  area  concerns. 

i Special  efforts  will  be  made  to  encour- 
age these  firms  and  to  give  them  the 
opportunity  to  compete. 


Contractor  will  assume  an  affirmative  obligation 
with  respect  to  subcontracting  with  small  busi- 
ness ...  ASPR  1-707-2 

a Designate  a liaison  officer/administrator 

• Consider  in  make/buy  decisions 

• Assure  equitable  opportunity  to  compete, 
particularly  by  actions  to  facilitate 
participation 

• Request  SBA  to  locate  sources  when  over 
$1  OK  and  no  small  business  sources  known 

• Incorporate  small -business  clause  in  all 
subcontracts  that  offer  substantial 
opportunity 

• S10K-$500K  - maximum  effort 

a Over  S500K  - establish  formal 

program 

• Notify  contracting  officer  of  awards  of 
S500K  or  more 


Although  our  compliance  with  Government  contracts 
is  critical,  it  is  our  total  company  performance 
in  all  divisions  of  the  company  that  tends  to 
establish  our  credibility  and  reputation. 
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Envi ronment 

High  level  and  high  degree  of  attention  in  all 
quarters. 

• Minority  and  small  business  pressure 
groups 

• NABM  - National  Association  of  Black 
Manufacturers 

• BCPNE  - Black  Corporation  Presidents 
of  New  England 

• LAMA  - Latin  American  Manufacturers 
Association 

• SBANE  - Small  Business  Association 
of  New  England 

i COSIBA  - Council  of  Small  and 

Independent  Business  Association 

• SBLC  - Small  Business  Legislative 
Council 

• Industry  pressure  groups 

• NMPC  - National  Minority  Purchasing 
Council  (private  sector  voluntary 
movement) 

• Congress 

• Black  Caucus 

• Committee  changes 

• GAO  report  - DOD  minority 

program 

t Legislation 

• Senate  Bill  2259 

t House  Bill  11318 

Senate  Bill  2259  (Senator  Gaylord  Nelson) 

• Establishes  written  small  business  sub- 

contracting plans  on  Government  procure- 
ments of  S500K  or  more 

• Requires  Government  agencies  to  review 

large  systems  procurements  and  take  steps 
to  subdivide  into  smaller  components 

• Empowers  the  SBA  to: 

• Obtain  statistical  data  directly 

from  prime  small  business  programs 

• Review  the  letting  of  subcontracts 

• Unilaterally  develop  the  small  busi- 
ness subcontracting  program  (drop- 
ping DOD  and  GSA) 

• Reduces  customer  management  of  prime 

small  business  programs  and  Interjects 
SBA 


House  Resolution  11318 
(Congressman  Joseph  Addabbo) 

• Replaces  many  draft  bills,  including 
HR  567,  the  Minority  Business  Act  of 
1977  (Parren  Mitchell) 

• Does  not  include  the  2 percent  or  5 per- 
cent requirement 

• Introduces  a "socially  and  economically 
disadvantaged"  definition,  which  means 
that 

• 51  percent  ownership  is  by  socially 
and  economically  disadvantaged 
groups  or  individuals,  including 
American  Black,  Hispanic  American, 
and  any  other  minority  or  individual 
found  to  be  disadvantaged  by  the 
Administration 

• Management  and  control  of  the  busi- 
ness is  by  such  person  or  persons 

• The  SBA  may  find,  on  basis  of  evi- 
dence, that  an  individual  group  is 
not  disadvantaged 

• The  SBA  may  add  others  to  the  dis- 
advantaged list 

• The  SBA  may  deny  assistance  if  it 
determines  that  competitive  via- 
bility cannot  be  achieved  within  a 
reasonable  period  of  time 

• Requires  written  subcontracting  plans 
for  procurements  of  $1,OOOK  if  for  con- 
struction of  any  public  facility  or 
$500K  for  any  other  purpose.  The  final 
plan  becomes  contractual. 


Environment 


Concerns 


Government  agencies 

• All  pushing 

• Increased  surveillance 

• Nonregulatory  proposal  requirements 

• Written  subcontracting  plan 
identifying  items  and  minority 
suppl iers 

• Goals 

Written  subcontracting  plans 
Breakouts 

Minority  business  base 
Reports  - Goals 

DOD/NASA  responsiveness  to  major  con- 
tractor problems  - help  needed 
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Cone! us  ions 

• Government  and  major  contractors  need  to 
reestablish  a "single"  program 

• Topside  pressure  is  not  an  effective 
approach  to  develop  a better  small/ 
minority  business  base. 

• The  approach  being  used  to  develop  the 
small /minority  business  base  may  not  be 
adequate 


CONTROLLING  THE  INDEPENDENT  PROFIT  MOTIVATED 
SUBCONTRACTOR 

Mr.  Peter  H.  Fowler 
Subcontract  Manager 
TRW 

The  Environment 


• High  ROI  - more  subcontracting 

• Stable,  predictable  business 

• Large  market  share  — only  the  first 
three  viable 

• Low  profile  - play  down  defense  and 
aerospace 

Out 

• Growth  as  an  independent  objective 

• Loss  leaders 

• Technology  for  its  own  sake 

• Paperwork 
Article  I - Scope  of  Work 

The  seller,  as  an  independent  subcontractor  and  not 
as  an  agent  of  the  buyer,  shall  

Duties  of  Management  of  Subcontractor 

• Act  always  in  the  best  interests  of  the 
stockholders 

• Make  a profit  {meet  ROI  target) 

• Maintain  the  organization 


ID!  IVll  IMS 


WTIVRTIN6  FACTOR 

tmer  cn 

ptwotwna 

*sro«  by 

MM 

wwiHia 
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0 (l/BOMTl  SO* 

KTUM 

0 cr  SPEC  MKNMU.V 

o wwna  risk 

0 ON*  Stic 
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o jECTMVia  socouu  l objccti«s 

0 KUO  WNtCTION 
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0 OVILCP  *« PUT ATICK 
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0 SUCTIW  CONTRACT  W» 

o nan  Man  HUNTS 

PIIIVIIY  OF  CJITWfl  HIP'S  IH  Wig  IBP 


Assure  What? 

Typical  Satellite 
Problems 

• Ground  testing  misses 
space  environmental 
problems 

• Test  data  overwhelms 
review  (problems 
missed) 

• Design  beyond 
experience 

The  Nature  of  the  Problem 

t Management  of  conflict 

• Clarification  of  roles 

• Recognition  of  humanity 

• Appropriateness  of  reaction 
The  Problem 

How  to  gain  assurance  without  taking  control. 


Typical  Response 
More  inspection 

More  data  taken 

Tighter  control 


CRITICAL  ELEMENTS  IN  SUPPLIER  CONTROL 


Are  We  Flowing  Down  the  Proper  Requirements? 


Marion  P.  Smith 

Manager,  Material  Quality  Assurance 
Honeywell  Inc. 


Provide  documentation  in  the  form  of 
procured  material  requirements  document 


Stipulate  for  each  class  of  item  being 
procured  or  each  unique  part: 


To  meet  mission  assurance  requirements  certain 
elements  are  critical 


Identify  and  place  the  essential  require 
ments  on  the  supplier 


Establish  and  maintain  a meaningful 
assessment  of  supplier  performance 


• Keep  the  loop  closed  on  supplier  correc 
tive  action 


What's  The  Requirement? 


Control  of  Purchases 


Responsibility.  The  contractor  is 
responsible  for  assuring  that  all  sup- 
plies and  services  procured  from  his 
suppliers  (subcontractors  and  vendors) 
conform  to  the  contract  requirements 


SAMPLE  OF  PROCURED  MATERIAL  REQUIREMENTS  DOCUMENT 


PURCHASE  MATERIAL  REQUIREMENTS 
FIGURE  1 

Purchase  Order — Quality  Requirements  Summary 
TYPE  OF  CONTRACT/OUALITY  SYSTEM  REFERENCES  - Check  as  Applicable 
DOD/MIL-Q-9858.  MIL-I-45208  l ' 1 

NASA/NHB  S300.4  (IB,  IC)  I 1 

NAVr/MIL-Q-21549  I 1 

OTHER  (SPECIFY)  l -J 


PROGRAM  NAME 


SAMSO  STD  73-5B 


Responsibility.  The  contractor  shall 
establish  a supplier  quality  assurance 
program  in  accordance  with  MIL-STD-1535 
and  this  standard 


ATTACHMENT  Q REFKI 


SPECIAL 


QUALITY  REQ ' TS 


PART  TYPE 


Purpose.  The  purpose  of  this  standard  is 
to  establish  the  procedures  for  an  effec- 
tive quality  assurance  program  for 
Government  procurement  involving  subcon- 
tracts when  MIL-Q-9858  or  MIL-I-45208 
and  this  standard  are  requirements  of  the 
prime  contract 


>1.  Deviation* 

Waiver*. 


Are  We  Verifying  That  the  Assurance  Requirements 
Are  Placed  on  Purchase  Order? 


Focus  on  Selected  Critical  Elements 


Perform  quality  assurance  review  of  procurement 
documentation  to  verify: 

• Proper  application  of  quality  assurance 
requirements  as  delineated  in  purchase 
material  requirements  document 


Are  we  flowing  down  the  proper 
requirements? 

Are  we  verifying  that  the  assurance 
requirements  are  placed  on  purchase 
order? 


• Use  of  approved  suppliers 


• Are  we  measuring  vendor  performance 
adequately? 

• Do  we  involve  the  subcontractor  ade- 
quately in  the  corrective  action  loop? 


Application  of  proper  requirements  for 
Group  I,  Group  II  and  Group  III 
purchases 


• 

Specific  requirements  to  be  placed 
on  the  subcontractor 

t 

Government  source  inspection 

• 

Contractor  source  inspection 

• 

Data  requirements 

• 

Certification  and  testing 
requirements 

• 

Packaging  and  shipping  requirements 

• 

Other  requirements 

SAMPLE  OF  APPROVED  SUPPLIERS  LIST 


JOINT  APPROVED  SUPPLIERS  LIST  0*/0*/T« 


(SORTED  BY  VENDOR  NAME  I 


FAMILY 

CODE 

MONfcVwELl  P*PT  NU^dE* 

CLASS 

code 

SUPPLIER  name  AND  ADDPeSS 

ouality 

level 

APPROVAL 

SRT  DTE  METH 

RES 

DTE 

remarks 

PSOCFS 

00-N-2B0  CL  l 2 

pos 

BROwARD  INDUSTRIAL  PLATERS 

FL 

LADFL 

PROCESS 

F 

711 

SF 

903 

• ADD#  1 

PBOCES 

OQ-P-35  TY  1 2 3 A 

pos 

BROWAPU  INDUSTRIAL  PLAT£PS 

FL 

ladfl 

PROCESS 

F 

Til 

SF 

903 

• ADD# 1 

PBOCES 

00-P-M6  TY  i 2 

POS 

BROWARD  INDUSTRIAL  PLATERS 

FL 

LADFL 

PROCESS 

F 

711 

SF 

903 

• ADD# 1 

PBOCf S 

OQ-S-365  TY  l 2 J 

POS 

BROWARD  INDUSTRIAL  PLATERS 

fl 

ladfl 

PROCESS 

F 

Til 

SF 

903 

• ADD#  1 

POOCFS 

"I l-"- 13508 

POS 

BRYSON  COATING  LAB  SAFETY 

harbor  fl 

PROCESS 

F 

612 

SF 

806 

PPOCFS 

H 1 1-<*.  4.520*0 

POS 

CINCH. MONADNOCK  city 

OF 

indjca 

*5208 

MCR 

TOT 

SM 

901 

10300  E valley  blvd 

PROCES 

FPS-18137  3*018752 

pos 

COMPRESSIVE  INDUSTRY  FT  LAUDERDAlFL 

PROCESS 

F 

60T 

SF 

801 

P»OCFS 

PPS-18137  E B «FLD 

POS 

comppessive  Industry  ft  laudeRDalfl 

PROCFSS 

F 

60T 

SF 

801 

PROCES 

M I L- l -68  66 

POS 

CO^AM  INSPECTION  INC 

MN 

PROCESS 

MNP 

606 

SM 

712 

1925  OAKCREST  SUITE  11 

PPOCFS 

PC-13*O*-O10 

Pos 

CONNOR  SPPING(,MFG  MONTEREY 

PK 

CA 

PROCESS 

MNP 

709 

SM 

903 

ATTN;  GAPY  SPANOS 

PBOCES 

OO-P-O350  TYPE  II 

Pos 

CONNOR  SPB.INGCMFG  MONTEREY 

OK 

CA 

PROCESS 

MNP 

709 

SM 

903 

831  MONTEREY  PASS  RO 

PPOCFS 

MJL-A-8623  TYPE  II 

POS 

cooperative  plating 

MN 

PROCESS 

MNP 

703 

SM 

809 

ATTNt  V.  ROSENBLUM 

PPOCfS 

PIL-C-2607* 

Pos 

cooperative  plating 

MN 

PROCESS 

MNP 

703 

SM 

809 

ATTN  V.  POSENBLUM 

PPOCFS 

HIL-C-55*!  8 

Pos 

cooperative  plating 

MN 

PROCESS 

MNP 

703 

SM 

809 

ATTN  V.  ROSENBLUM 

PPOCFS 

MIL-T-10727  TYPE  I 

Pos 

cooperative  plating 

MN 

PROCESS 

MNP 

703 

SM 

809 

1605  IGELHART  AVE, 

PPOCFS 

PC-H3J1-01 

POS 

cooperative  plating 

MN 

PROCESS 

MNP 

703 

Sm 

809 

ATTN#  V,  ROSENBLUM 

Pos 

cooperative  plating 

MN 

PROCESS 

MNP 

703 

SM 

809 

ATTN  V,  ROSENBLUM 

POS 

cooperative  plating 

MN 

PROCESS 

MNP 

703 

SM 

809 

ATTN  V.  ROSENBLUM 

POS 

CRAFTMAN  PLAT  6 TINNING 

PROCESS 

MNP 

705 

SM 

811 

• ADD#1 

POS 

D » ■ * c Pi  ATJnG  InC  31*0  ** 

AVE 

N SFL 

PROCESS 

F 

608 

SF 

002 

INC  31*0  *6 

AVE 

N SFL 

PROCESS 

F 

608 

SF 

' 31*0  *6 

AVE 

N 5FL 

PROCESS 

F 

* * * 

* *6 

AVE 

N SFL 

PRf>rr«- 

• AUP  " 


Are  We  Measuring  Vendor  Performance  Adequately? 

An  organized  vendor  rating  syst  it  is  required  to 
give  perspective  on  vendor/subcontractor 
performance. 

Ratings  should  reflect: 

• What  has  he  done  lately? 

• What  is  his  past  year's  performance? 

• What  is  the  cost  of  the  material?  Of 

rejections? 

Vendor  rating  reflecting  cost  impact  of  nonconform- 
ing material  is  very  useful. 

Vendor  Rating  Examples: 

• Lot  acceptance  percentage 


• Lot  acceptance  percentage  for  latest 
month 

• Lot  acceptance  percentage  for  last 
12  months 

• Parts  acceptance  percentage  for  last 
month 

• Parts  acceptance  percentage  for  past 
12  months 

• Cost  of  material  procured 

• Cost  of  rejections  - Actual  $$  and  cost 
impact  (rejection  cost  per  $1000  of 
procured  material) 
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ELECTRICAL  SUPPLIER  QUALITY  RATING  REPORT 


*P<»  FLIX  CABLE S/PR  IN TED  CB 


FIRST  LINE  • «*«CH 
SECOND  LINE  • LAST  MONTHS 


SOURCE 

CODE 

VENDOR 

NAME 

TOTAL 

LOTS 

ACPT  *ACPT 

TOTAL 

PARTS 

ACPT 

6ACPT 

PROC. 

COST 

IMPACT 

COST 

C.I, 

ANYCXO 1 

APPLIED  DIGITAL  DATA  SYS 

0 

0 

0 

0 

0 

0 

0. 

0, 

0, 

2 

2 

100 

2 

2 

100 

680,00 

0. 

0, 

BHADJ01 

BOUDREAU  CONSULTING 

0 

0 

0 

0 

0 

0 

0, 

0, 

0, 

1 

1 

100 

2 

2 

100 

626.00 

0, 

0. 

t)N02i04 

barg*le  industries 

0 

0 

0 

0 

0 

0 

0, 

0, 

0, 

l* 

11 

tc 

60 

90 

• 3 

4074,26 

330,00 

81,0 

CCAAVOl 

CIRTtL  INC, 

0 

0 

0 

0 

0 

0 

0, 

0, 

0, 

12 

9 

79 

1TS3 

• 91 

90 

26063,00 

900,00 

10,7 

CC00301 

COLORADO  CIRCUITS 

0 

0 

0 

0 

0 

0 

0, 

0, 

0, 

1 

1 

100 

2 

2 

100 

986,24 

o. 

0. 

CIA1301 

COLLINS  RADIO  CO 

1 

0 

0 

36 

0 

0 

1416,60 

120,00 

• 4,7 

16 

9 

96 

996 

24| 

44 

18979,39 

•30,00 

44,7 

0FL0J03 

GENERAL  COMPONENTS 

2 

2 

100 

19 

19 

100 

1330,00 

0. 

o. 

2 

2 

100 

19 

19 

100 

1930,00 

0, 

0, 

OFL0402 

GENERAL  COMPONENTS  INC, 

16 

19 

92 

21 

20 

99 

•780,00 

120,00 

19,7 

99C 

900 

•9 

1079 

2690 

•6 

979108,74 

4940.00 

11.9 

HMA0J61 

HONEYWELL  INfO  SYSTEMS 

0 

0 

0 

0 

0 

0 

0, 

0, 

0, 

6 

6 

100 

16 

14 

100 

4697,00 

0, 

0, 

39 

36 

AT 

1A60 

1S4C 

•1 

34180,40 

996,00 

16,3 

ICC 

707 

T9 

17106 

9010 

93 

•16911,14 

30922,00 

37,9 

Do  We  Involve  the  Subcontractor  Adequately  in  the 
Corrective  Action  Loop? 

Communicate  with  subcontractors  quality  organiza- 
tion regarding  nonconforming  material 


a Information  to  subcontractor  regarding 
nature  and  quantity  of  nonconformances, 
requirements  for  response  to  customer  and 
information  on  who  he  can  contact  in  the 
contractors  organization  for  details. 


Honeywell 


Attn:  Quality  Control  Manager 


Attachment:  Supplier  Nonconformance  Report  # 150 

Dear  Sir: 

Material  recently  received  at  Honeywell  Avionics  Division  was 
found  by  our  Incoming  Inspection  Department  to  contain  nonconforming 
characteristics.  These  nonconformances  are  defined  on  the  attached 
Supplier  Nonconformance  Report. 


Mission  assurance  success  is  enhanced  if  we: 

• Identify  the  proper  quality  requirements 
for  the  subcontractor 

• Verify  that  these  requirements  are  con- 
veyed to  the  subcontractor 

• Rate  vendors  on  an  organized  basis  for 
timely  action 

• Communicate  fully  with  vendors  for  cor- 
rective action. 


WORKSHOP  SUMMARY 


COMMENTS  BY  JEROME  FISHEL,  CTI , 
AS  VIEWED  BY  A SECOND  TIER 
SUBCONTRACTOR 


Subcontractor/Supplier  Interface  and  Control 

• Problem  No.  1 

Lack  of  definition  and  standardization  of  technical 
requirements  and  consistency  of  flow  down  to  sub- 
contractor level. 

Recommended  Solution: 

Establish  standardization  board  with  the  authority 
to  publish  internal  direction. 

Action  Agency  SAMSO 

• Problem  No.  2 

Lack  of  bidders  due  to  economic  conditions. 
Recommended  Solutions 

• Recognize  front  end  funding  and  add 
inducement  to  bidders  to  bid 

4 Tailor  requirements  when  practical  to  fit 
small  business  and  minority  capabilities 

• Recognize  development  funds  for  small 

business  and  minority  business  to  develop 
technological  capabilities. 

Action  Agency  S BA/ SAMSO 

• Problem  No.  3 

Technical  requirements  too  detailed. 

Recommended  Solution 

• Emphasize  feasibility  studies 

t Emphasize  performance  type  requirements 

• Minimize  "how  to"  specs. 

Action  Agency  SAMSO/DOD 

• Problem  No.  4 

Lack  of  failure  historical  data. 

Recommended  Solution 

Broader  dissemination  of  failure  data  throughout 
Industry  and  government. 

Action  Agency  - SAMSO 

e Problem  No.  5 

Too  many  audits. 

Solution 

Interchange  among  agencies  and  programs  to  minimize 
audit  frequency  and  duration. 

Action  Agency  - SAMSO/DOD/ NASA 


1 .  Chain  of  Command: 


When  we  request  a deviation  from  our  Customer, 
we  are  often  told  that  they  are  reluctant  to 
take  it  Upstairs  because  of  the  particular 
situation  which  may  exist.  Therefore,  we  are 
denied  approval  of  our  request  without  regard 
to  its  merit.  I would  suggest  that  some 
method  be  developed  to  allow  us  to  go  directly 
to  higher  authority  in  those  special 
situations. 


2.  Small  Management  Company  Briefings: 


I believe  SAMSO  should  acquaint  the  Manage- 
ments of  their  small  company  team  members  with 
the  full  details  and  significance  of  the  vari- 
ous programs.  I came  away  from  our  meeting 
with  a greater  sense  of  the  appreciation  and 
importance  of  what  we  are  doing. 


3.  Technical  Expertise : 


I stronqly  recommend  that  the  SAMSO/ Aero  Space 
Technical  Staff  allow  the  subcontractor  to 
produce  the  parts  using  methods  and  processes 
which  they  are  trained  and  familiar  with.  It 
is  necessary  in  many  cases  for  a Manufacturer 
to  be  'monitored'  to  insure  that  he  complies 
with  good  practices.  However,  this  is  differ- 
ent than  being  directed  in  how  he  should  do 
it. 


4. 


It  would  be  desirable  if  SAMSO/Aero  Space 
would  provide  technical  services  such  as: 
failure  analysis,  specification  testing,  etc. 
to  the  smaller  subcontractors  where  necessary. 
This  would  be  very  helpful. 
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WORKSHOP  F 

DEVELOPMENT  & CONTROL  OF  MICROELECTRONICS/HYBRIDS/LSI 
DEVICES  FOR  SPACE  APPLICATION 


Co-Chairman 

Dr.  Jack  Hilibrand,  Staff  Tech.  Advisor 
Gov't  Systems  Div.,  Engineering 
RCA  Corporation 


Mr.  James  J.  Egan 

Head,  Components  & Product 

The  Aerospace  Corporation 


AGENDA 

Wednesday,  April  26 

0830-0840 

Opening  Remarks 

Co-Chairmen 

0840-0905 

CMOS/SOS  Reliability  Status 

Mr.  Eugene  M.  Reiss,  RCA 

0905-0930 

I^L/LSI  in  Military  Applications 

Mr.  Klaus  K.  Schuegraf,  Northrop 

0930-0955 

Radiation  Effects  on  LSI 

Mr.  James  P.  Raymond,  Mission  Research 

0955-1020 

Hi  Rel  Applications  of  Custom  LSI 

Mr.  John  D.  Heightly,  Sandia 

1020-1045 

Design  Tools  for  Custom  LSI 

Mr.  Jack  Hilibrand,  RCA 

1045-1110 

Testing  Microprocessors 

Mr.  Lenward  E.  Holmess,  HAC 

1110-1135 

Microprocessor  Slash  Sheet 
Development 

Mr.  W.  Richard  Scott,  JPL 

1135-1200 

Memory  Design  and  Testing  for  Hi 

Rel  Applications 

Dr.  Joseph  S.  Bravman,  Fairchild 

1200-1215 

Class  S LSI  Plans 

Mr.  Alan  J.  Carlan,  Aerospace 

1215-1315 

Lunch 

1315-1340 

Hybrid  Microcircuits 

Mr.  Jerome  Fishel,  CTI 

1340-1406 

Hi  Rel  Hybrid  Manufacturing 

Mr.  Ralph  Redemske,  Teledyne 

Considerations 

v«£i9bE 

1405-1430 

Hybrid  Chip  Screening  and 

Evaluation 

Mr.  John  deJong,  JPL 

1430-1455 

Class  S Hybrid  Circuit  Committee 

Mr.  Donald  L.  Fresh,  Aerospace 

1455-1520 

Implementing  Commanders  Policy 
for  Supplier  Surveillance 

Mr.  Warren  Geller,  LMSC 

- f.  ' ; „ 

1520-1545 

Coordinated  Procurement  for  IUS 

Mr.  Wilber  L Barker,  Boeing 

1545-1600 

Break 

1600-1715 

Conclusions  and  Recommendations 

Group  Discussion 

1715-1730 

Summary 

Co-Chairmen 

1730 

Adjournment 

: ..  ■' 

, 

mm 
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CMOS/ SOS 

A LSI/VLSI  TECHNOLOGY 
Eugene  M.  Reiss 

Mgr.,  MOS  High  Reliability  Engineering 
RCA  Solid  State  Division 


The  benefits  of  CMOS  are  well  known  throughout  the 
industry.  As  a result  there  has  been  dramatic 
growth  in  the  number  of  CMOS  functions,  devices 
delivered  and  the  number  of  vendors  during  the  past 
six  years.  Coupling  the  CMOS  technology  to  a sap- 
phire substrate,  CMOS/SOS,  produces  additional 
improvements  in  performance  and  cost  for  LSI/''LSI 
functions.  These  improvements  are  related  to 
properties  inherent  in  the  silicon  on  sapphire 
structure,  such  as:  a)  low  parasitic  capacitance, 
b)  high  packing,  c)  density,  d)  good  yield  poten- 
tial, e)  resistance  to  transient  effects.  However, 
there  are  concerns  which  relate  to  a)  high  cost  of 
starting  wafers,  b)  reliability  of  a new  tech- 
nology, c)  production  capability  and  d)  multiple 
sourcing. 

Although  the  SOS  technology  has  been  in  the  labora- 
tory for  over  10  years,  it  is  during  the  past  two 
years  that  it  has  been  converted  to  a production 
process.  As  a result,  the  cost  of  the  sapphire 
substrate  has  been  decreasing  and  additional  major 
cost  reductions  are  anticipated  through  the  intro- 
duction of  the  ribbon  process.  As  this  occurs,  the 
cost  of  the  substrate  in  the  finished  wafer  will 
also  decrease  and  will  not  be  a significant  cost 
factor.  During  this  same  period  of  time,  produc- 
tion capability  was  established  and  long  term 
device  stability  at  125°C  demonstrated.  As  a 
result  RCA  has  committed  all  new  CMOS  LSI/VLSI 
designs  to  the  SOS  technology.  The  ability  to  pro- 
duce SOS  devices  by  multiple  vendors  has  also  been 
demonstrated.  The  recent  RCA-Intel  technology 
exchange  agreement,  will  provide  another  major 
source  of  SOS  products. 

Although  CMOS/SOS  has  these  advantages: 

t Low  parasitic  capacitance 

• High  packing  density 

• Reduced  sensitivity  to  effect  of 
pinholes 

• No  substrate  warpage  during  diffusion 

• Built-in  resistance  to  transient  radia- 
tion effects 

• Military  temperature  range 

• Wide  operating  voltage. 

CMOS/SOS  Advantages  for  LSI/VLSI 

Lower  parasitic  capacitance  results  In  greater 
speed  and  lower  operating  power 

t = RC 


STRUCTURE  OF  BULK  MOS  TRANSISTORS 


STRUCTURE  OF  SOS  MOS  TRANSISTORS 


COMPARATIVE  DISSIPATION 
BULK  C04040  VS  SOS  CDS4040 
(12  STAGE  COUNTER! 


CMOS/SOS  Advantages  for  LSI/VLSI 
• High  packing  density 

s Guardbands  not  required 

• Planar  structure 

• Capability  for  scaling 
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UPDATED  MOS  LSI  TECHNOLOGY  OVERVIEW 


PARAMETER 

H-MOS 

1977 

SCALED 

DOWN 

H-MOS 

1980+ 

V-MOS 

1977 

SOS 

1977 

CDP1804 

SOS 

1978 

LAYOUT  DENSITY 
IGATES/MM*) 

170 

200 

220 

150 

242 

SPEED  POWER  PRODUCT 
(PJ) 

1 

0.2 

1 

0.2 

GATE  DELAY  |ns) 

1 

0.4 

1 

0.5 

NUMBER  OF  THIN  FILMS 

2 

2 

3 

3 

NUMBER  OF  IMPLANTS 

3 

3 

3 

2 

"■  • From  "Electronics"  • • • August  18,  1977  • • • Page  99 


IK*  1 CMOS/SOS  RAM 
AEROSPACE  APPLICATION 
SCREENING  DATA 


OATt  CROSS  NO 
C00<  Of  QUITS 


INITIAL  IN  HR 
TIST  OYN 

_ 'V 


14  HR  24  MR. 
SI  SM 

f»J  <\1 


fU  BURN 
140  HR  240  HH 
OVN  I OVN  II 

ISI  _IM 


TIST 
SS  C 


IS) 


7738  394 

7739  70 

7741  169 

774 2 166 

7743  160 

7743  242 

7746  366 

7747  302 

7781  1MO 


63  94 

68  92 

65  71 

45  64 

69  66 

68  87 

77  89 

74  87 

77  87 


97  99 

TOO  98 

TOO  100 

100  100 

99  TOO 

99  99 

99  99 

96  98 


100  100  96 

79  100  100 

100  100  98 

96  100  86 

98  100  100 

97  100  100 

TOO  TOO  10 0 

97  99  94 

97  99  99 


TOTAL  3019 


67  85  99 


96  99 


97  99  98 


IKxl  CMOS/SOS  RAM 
AEROSPACE  APPLICATION 
RELIABILITY  DATA 


T • 125’C 

SCREENED  DEVICES 


LOT 

LIFE  TEST  SAMPLE 
VOLTAGE  SIZE 

FUNCTIONAL 
failures 
1000  HRS. 

TOTAL 

HOURS 

TESTEO 

FUNCTIONAL 

FAILURES 

1 

10 

12 

0 

3624 

0 

2 

10 

19 

0 

3288 

0 

3 

to 

18 

0 

3624 

0 

4 

10 

14 

0 

5802 

1 (5466  MR) 

5 

10 

45 

0 

NOT  TESTEO 

6 

5 

21 

0 

3316 

0 

7 

5 

16 

0 

3316 

0 

8 

5 

8 

0 

3316 

0 

• 

5 

13 

0 

5730 

0 

10 

5 

77 

0 

TOTAL 

243 

0 

TOTAL  DEVICE  HOURS  597.794 
FUNCTIONAL  FAILURES  1 

FAILURE  RATE.  60%  CONFIDENCE 


T - 12S*C  - 0.34V7000  HRS. 

T • S6*C  • Q.0003%/1000  HRS. 


Concerns : 

• High  wafer  costs 

• Reliability  of  a new  technology 

• Production  capability 

• Multiple  sources 
CMOS/SOS  Concerns 


• High  wafer  costs 

• Sapphire  finished  wafer  cost  ratio 
to  bulk  silicon  is  less  than  two  to 
one  today  and  improving 

• Higher  packing  density  gives  60  per- 
cent more  dice  on  a sapphire  wafer 

• Improved  yields  are  expected  to 
result  in  lower  die  cost  than  with 
bulk 

• As  a result,  all  new  LSI  and  VLSI 
CMOS  designs  at  RCA  are  being  imple- 
mented in  SOS  technology. 


WAFER  COSTS 


CMOS/SOS  Concerns 


• Reliability  of  a new  technology 

• Commercial  product  data  base 
established 

• By  end  of  1977  over  150,000  net 
die  per  month 

• 0.0004  percent/1000  hrs,  55°C, 
60  percent  confidence 

• Screening  data  indicates  product 
stability 

• High-reliability  data  base 
established. 
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INDUSTRY  MOS  LSI  TECHNOLOGY  OVERVIEW 


fNOM  ILICTHONlCf 
AfNIL  1 l»f* 
*AG|  /• 


TICHNOLOOV 


riAIOMMANCI  CO*T 

KtOPAtJATlON  POMtROlLAV 

Of  LAV  PROOUCT  OfNSITV  CHIf  Sl/I  .Mtil  I 

(ml  (RJI  (MVtCfSMM*l  lOATIMHMl 


HIGH  - THAI SMOlO 
9 CHANMl  MCTAL  OATf 
9 CHANNIL  SILICON  GATf 
N CHANNIL  SILICON  GATf 
\ CHANNIL  SILICON  GAM 
Din  It  ION  LOAD 
N CHANNIL 
OOOSLl  90LVSIIICON 
SILICON  GATI  C MOS 


10 

IS 

ia 

«o 

10 


V MOS 
0 MOS 
•Of/C  MOS 
l*L  IOOUOLI  LIVfLI 


S 10 

I - I 01 

S - SO  001  I 


ISO  SO  1 a 1 

ro  so  ss>ss 

70S  OS  III 

170  107  0 • 0 

sas  m *.* 

110  «S  SSaSS 

MO  ITS  - 

660  an  Sat 

MO  IM  SS 


CMOS/SOS  Concerns 


t Multiple  sources 

t NSA  study  demonstrated  that 

5 vendors  had  capability  to  process 
an  SOS  test  chip  with  same  resultant 
test  data 

# Hewlett  Packard 

• INTEL  agreement. 

Conclusions 

1.  The  CMOS/SOS  technology  has  made  signifi- 
cant advances  during  the  past  two  years 
and  it  is  in  production  status  at  RCA  and 
can  be  qualified  for  applications  in 
space. 

2.  CMOS/SOS  is  the  leading  technology  for 
VLSI  functions  as  a result  of  its  out- 
standing speed  power  product  and  high 
packing  density. 

3.  CMOS/SOS  devices  have  demonstrated  long 
term  stability  at  125°C. 

4.  Mask  compatible  second  sourcing  is 
available. 

5.  RCA  is  fully  committed  to  supplying  space 
qualified  CMOS/SOS  parts. 

Recommendations 


The  aerospace  industry  should  be  making  more  use 
of  CMOS/SOS  technology. 


I 
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i2l-lsi  in  military  and  space  applications 

Klaus  K.  Schuegraf 
Mgr.,  Integrated  Electronics  Lab 
Northrop  Research  and  Technology  Center 


SUMMARY 

The  military  application  of  LSI  circuitry  places 
increasing  demands  on  the  present  technologies.  A 
major  requirement  for  future  systems  implemented 
with  digital  random  logic  is  to  increase  the  level 
of  integration  on  a single  LSI  chip  such  that  the 
component  count  be  increased  from  a few  thousand 
(i.e.,  LSI)  to  over  ten  thousand  (i.e.,  VLSI,  Very 
Large  Scale  Integration).  Past  performance  of 
military  systems  that  utilize  MSI  and  LSI  elec- 
tronic components  exhibit  improved  reliability  and 
in  many  cases  lower  costs  for  initial  procurement 
as  well  as  for  maintenance.  These  advantages  will 
be  extended  by  the  application  of  VLSI  for  custom 
random  logic  for  military  applications.  However, 
this  increase  in  integration  level  should  be 
achieved  without  a substantial  decrease  in  the 
ability  of  the  device  to  operate  over  the  full 
military  temperature  range  or  in  nuclear  radiation 
environments. 

o 

Integrated  Injection  Logic  (I  L)  is  a high  packing 
density  technology  that  utilizes  the  structural 
merging  of  an  npn  transistor  with  a pnp  current 
source  to  provide  moderate  performance  at  very  low 
voltage  operation.  Thus,  power-delay  products  of 

0.5  x 1 0~ * 2 j are  realistic  for  gates  using  a set 
of  "relaxed"  minimum  geometries. 

2 

I L LSI  technology  is  well  suited  to  a wide  range 
of  military  digital  logic  applications.  These 
applications  frequently  demand  survival  in  nuclear 
radiation  environments.  The  low  power  and  high 
packging  advantages  of  I 2l  need  not  be  "traded- 
off"  for  improved  radiation  hardness  levels. 

Design  and  performance  parameters  for  an  I?L  tech- 
nology for  military  applications  are: 

1.  Logic  Gate  Performance  Parameters 

• Operational  Temperature  Ranqe: 

-550  to  1 25°C 

• Minimum  Propagation  Delay: 

5-10  ns/gate 

• Power  Consumption  at  Minimum  Delay: 

0. 1-0. 3 mW/gate 

• Logic  Gate  Packing  Density: 

250  - 500  gates/mm? 

2.  Radiation  Failure  Levels 

1 3 2 

t Neutron  Fluence:  <5  x 10  n/cm 

• Total  Ionizing  Dose: 

5 x 106  rad(Si) 

• Ionizing  Dose  Rate: 

5 x 109  rad(S1 )/sec 

These  performance  parameters  are  consistent  with 
realistic  electronic  systems  requirements  of  the 
1980's.  They  can  support  the  majority  of  hardware 


requirements  that  the  military  presently  implement 
with  small-scale  and  medium-scale  TTL  devices. 
However,  the  use  of  large  scale  I?L  devices  has  the 
advantage  of  lower  power  consumption  and  higher 
packing  densities  than  TTL.  Hence,  a radiation 
hardness  I^L  technology  offers  military  electronics 
systems  suppliers  a means  to  keep  pace  with  the 
demand  for  increasing  functional  complexity  without 
sacrificing  radiation  hardness. 

Trend  in  Electronic  Hardware  for  Military  and  Space 
Applications 

• Increasing  use  of  digital  techniques 

t Data  transmission:  secure  voice, 

AJ  data  links 

• Digital  signal  processing:  video, 

radar,  BW  compression 

• Avionics/spaceborne 
computers/controllers 

• Increasing  sophistication  made  possible 
by  low  cost  microprocessor  chips,  low 
cost  memory 

• Demand  for  military  qualified  LSI 
technology 

• 10^  - 10^*  functions  per  LSI  chip 

at  moderate  cost 

• Medium  speed  at  low  power 

• Full  MIL  temperature  range 

• Radiation  hardened 

What  LSI  Technology  to  Choose? 

Problem:  No  single  LSI  technology  exists  to 
satisfy  the  majority  of  mi litary/space  requirements 

• Low  power  at  medium  speed:  100  uW/gate 
at  25  MHz 

• MIL  temperature  range:  -55  +125°C 

• Radiation  hardened:  Total  dose 

y > 10^  Rads 

Neutron  ~ 

4 > 10^3  n/cirr 

Transient  upset 
y > 109  Rads/sec 

• Moderate  cost:  •vlC  per  gate 

• Multiple  sourcing 

• High  reliability 

2 

I L/LS?  Technology  is  Ideal  for  Complex  Logic  Chips 

• Adequate  speed  (15  to  20  MHz) 

• Low  power  (1 00  uW/gate) 

• High  packing  density  (5000  gates/chip) 
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PRECEDING  paoC 


■«ia  i m mm*,  i , — 

BUNK-NOT  FILMED 


PROPAGATION  OOAY  Tl 


Full  military  temperature  range  (-55  to 
+125°C) 


COMPARISON  Of  LSI  POWER  CONSUMP  TION 


Simplified  fabrication 


TECHNOLOGY 


POWER  FOR  ltr  CATES 
(WATT) 


Good  radiation  hardness 


Growth  potential 
Speed  (30  MHz) 

Density  (>10,000  gates/chip) 

Chip  compatibility 

Analog  and  digital  functions 

2 

High  speed  ECL  and  medium  speed  I L 


SOS /CMOS 


ASSUMPTIONS:  111  10  MHz  OPERATION 

12)  FOR  SOS/CMOS,  25%  Of  ALL  GATES  OPERATE 
AT  10  MHz 


npn 

COLLECTORS 


BASIC  I I LOGIC  GATE 


NUMBER  Of  COMPONENTS  PER  CHIP  VS  TIME 


A - BINARY  COUNTER 

I - RING  OSCIUATOR  - 
HIGH  SPEED  PROCESS 


TTCHNOIOGY 


I io  no  Sooo 

POWER  DISSIPATION  PER  GATE  IpWI 

MEASURED  AND  ESTIMATED  DELAY-POWER 
PERFORMANCE  FOR  l2L  GATES 


l?UPWSCNTI 

lira6 

JaIO9 

1*1  i»nv»NCTDi 

1.10“ 

SilO6 

Sr  IO9 

m 

1 1 IO5  - 1 . IO7 

i > io*  ■ i . n* 

(CL 

LB.'5 

1 . w'  l . 10* 

1 a W*  - 1 1 10* 

sos  raws 

i«»“ 

1 * ID5  - 1 « to‘ 

! . »*  ■ 1 1 10 
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NIMAHKS 


riRIOKMANU  CRIURIA 

l?l  It Rf ORMAN)  1 

REMARKS 

fUi  Mil  ILMPERAIURE 
RANCE 

• MAXIMUM  SPUD  INSENSIIIVf  to 
IIMPERAIURES  OVER  to  M»*C 

• MAXIMUM  SPUD  DEPENOS  ON  CURRENT 
AVAILABLE  IO  SUPPLY  npn  BASE  CHARCE  - 
BOTH  DLPEND  ON  IEMPERAIURE  IHROUGII 
n,  HENCE  IEMPERAIURE  EIUCIS  CANCIl 

out 

SPUD  POWER 

IRAK  OHS 

• SPUD  IOR  l\  IS  VARIAILE  WIIH 
INJECIOR  CURREN1  OVER  4 0ECADIS 

• POWER  IOR  VARIOUS  COMPONENIS  0*  A 
SUBSYSTEM  UN  BE  ADJUSIID  10  ACHIEVE 
REQUIRED  SPEED 

• SINGLE  GENERIC  DEVICE  TYPE  tr|  . MICRO 
PROaSSOR.  RAM.  Mil  ARRAY.  Me.  1 CAN 

BE  DESIGNED  IOR  VARIETY  Or  APPEIMIIONS 

INPUI 1 llUIPUt 
COMPAIIDIIITY 

• 1*1.  A BIPOLAR  11  CMNOl  OCY,  OUERS 

m ano  ta  compaiibhiiy 

• HR  HIGH  CURRENT  MIN  « BIPOLAR  IRAN 
SISIORS  ALLOWS  "SCALING  UP"  01  CURRENIS 
WITHOUT  IXCfSSItf  IIMI  DllAY  OVER  SIV- 
ERAL  SIAGES  AS  IN  MOSUSI 

RADI  AIION 

HAR0CNING 

• l\  WIIRS  1HE  POTENTIAL  IOR  COOO 
HARDNESS  At  INCREASED  LOGIC 

SPUDS 

• THE  PACKING  DENSITY  ANO  COSI  BUKIIIS 

OT  COMMERCIAL  l7t  WILL  Not  BE  LOSI  IOR 
HARDEN  0 l\ 

SUMMARY  OF  |2L  PtRFORMANCt  CH  ARACTERISIICS 


j5  RUIXBIUTY 


BENEFITS  OF  SCHOTTKV  BASE  l?L  FOR  RANDOM  LOGIC 

• SIMPLIFIED  LSI  DESIGN  AND  LAYOUT 

- MULTIPLE  GATE  INPUTS  AND  OUTPUTS 

- REDUCED  INTERCONNECTION  COMPLEXITY 

- IMPROVED  ISOLATION  BETWEEN  CELLS 

- GROWTH  POTENTIAL  FOR  VLSI 

• GOOD  MINIMUM  DELAY  TIME  <5nsec 

• FAVORABLE  DELAY -POWER  PRODUCT  <0.5  pJ 

• EXCELLENT  RADIATION  TOLERANCE  <J»5  x 1013n/cm2 

Y>5  x 106Rads 
Y>5  x lO^Rads/sec 

• IMPROVED  INPUT/OUTPUT  INTERFACE  CAPABILITY 

TTL  COMPATIBLE 


PtRFORMANCt  AOVANIACtS:  LOW  POWtR  ItCHNOLOGY  LOW  CHIP  IlMPtRAIURt.  LOW 
CURRCN!  MNSITItS 

LOW  VOllACf  OPtRAFION  l<  1 VI  - RtOUCtO  EITCTROLYTIC  EFFICTS 
HIGH  SURFACt  CONCtNIRAIIONS  - NO  SURFACE  INVERSION 


IESI  RESULTS: 
ELECTROLYSIS: 

BIAS  TEMPERATURE: 

BIAS  HUMIDIFY: 
25°CISO»RH 

GAIN  DEGRADATION 

ACCUMULATED 
DEVICE  HOURS 


>1000  HR.  OPERATION  IN  SALT  WATER 

<l.0FirAT60°C 

<10  FITS  OVER  40  YEAR  LIFE 

>IO»M250 ...m°C 

EOUIVAUNT  TO  60  YEARS  END-Of -LIFE 

DEGRADATION 

60  < ID6  HRS.  WITH  NO  REPORTED  CHIP  FAILURES 


SOURCE:  F.W.  HEWLETT  6 R A PEDERSEN 
I4lh  PROC.  REl.  PHYS  1916 


- TRI-STATt  CAPABILITY 


Conclusions 

2 

• I L is  a viable  LSI  technology  for 
military  and  space  applications 

2 

• I L can  meet  temperature  and  radiation 
hardness  requirements  for  space 
applications 

2 

• I L gate  arrays  will  permit  low  volume/ 
fast  turn-around  LSI  implementations 

2 

• I L failure  rates  of  less  than  10  fits 
under  normal  stress  are  predicted 

2 

• Advanced  I L has  growth  potential  for 
VLSI  ( >50  MHz,  >104  gates  per  chip) 

Recommendations 


• Provide  incentives  for  LSI/VLSI  utiliza- 
tion in  space  systems 

• Encourage  set-up  of  systems-oriented 
custom  LSI  centers  with  qualified  low 
volume  fabrication  facilities 

• Encourage  develoDment  of  LSI  technologies 
with  high  degree  of  fabrication  com- 
monality to  assure  second  sourcing 

• Establish  procurement  policies  to 
guarantee  lifecycle  support  for  LSI  parts 

• Develop  user-oriented  LSI  performance- 
and  reliability-test  procedures  and 
specifications. 


ADVANCED  I ZL  TECHNOLOGY:  SCHOTTKY  BASE  I2 
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RADIATION  EFFECTS  IN  LSI 


James  P.  Raymond 
Technical  Staff 
Mission  Research  Corporation 


ABSTRACT 

Two  critical  aspects  of  radiation  effects  on  LSI 
arrays  are  presented  for  discussion:  1)  the  rela- 
tive evolution  of  high  performance  LSI  arrays  and 
radiation  hardness,  and  2)  unique  experimental  and 
analytical  techniques  necessary  for  array  evalua- 
tion and  hardness  assurance. 


Radiation  susceptibilities  of  the  major  LSI  tech- 
nologies (n-MOS,  CMOS/SOS,  and  I?L)  are  briefly 
reviewed.  In  each  case  the  overall  radiation 
susceptibility  increases  with  increasing  array 
complexity,  decreasing  logic  cell  energy,  and 
decreasing  the  design  margin  of  critical  internal 
parameters. 


The  nature  of  radiation  effects  imposes  unique  test 
requirements  on  LSI  arrays.  The  basic  requirement 
is  for  a comprehensive  evaluation  of  array  perfor- 
mance, particularly  for  arrays  designed  without 
consideration  of  radiation  effects.  Any  logic  cell 
or  sub-circuit  can  lead  to  array  failure.  A more 
difficult  requirement  is  the  definition  of  a set  of 
test  vectors  to  establish  worst-case  bias  during 
radiation.  For  MOS  arrays  the  electrical  bias  dur- 
ing exposure  is  a first-order  effect  in  determining 
the  total-dose  failure  level.  For  any  digital 
array,  the  logic  upset  level  is  a strong  function 
of  the  logic  state  during  pulsed  ionizing  radiation 
exposure.  In  each  case  a large  number  of  bias  con- 
ditions can  be  defined  and  techniques  must  be 
developed  to  identify  the  minimum  set  of  test  vec- 
tors sufficient  to  reveal  the  worst-case.  Short- 
term annealing  effects,  critical  in  some 
applications,  impose  a time  urgency  on  electrical 
performance  characterization  following  radiation 
exposure. 
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Long-Term  Ioniza- 
tion Damage 
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Transient  Logic 

Upset  Level 
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0 

0 

* 

♦ ♦ 

♦ ♦ superior,  ♦ good,  0 average,  - below  average,  - - weak 
Subjective  Comparison  of  LSI  Technologies 


LIMITING  RADIATION  EFFECTS 

n - H 0 S / L S I (1,2) 

TOTAL  IONIZING  DOSE:  105  - 102*  rads(Si) 

CMOS/SOS/LSI  l3) 

TOTAL  IONIZING  DOSE:  104  - 105  rads(Si) 


NEUTRON  FLUENCE:  1015  - 1014  n/cm2  (1  MeV  eouiv.) 


Efforts  in  the  development  of  diagnostic/test  sup- 
port techniques  also  include  the  use  of  laser 
excitation,  test  elements  and  coupons,  and  con- 
siderations in  overall  evaluation  of  parameter 
margin  from  non-destructive  techniques.  It  is  not 
clear  that  these  efforts  are  capable  of  either 
cathing  the  state-of-the-art  in  LSI  technology,  or 
keeping  pace  with  its  evolution. 


(1)  Myers,  NS-23,  No,  6,  pp.  1732-1737;  Dec.,  1976. 

(2)  Myers,  NS-24,  No.  6,  pp.  2169-2172;  Dec.,  1977. 

(3)  Lee,  et.  al.,  NS-2A,  No.  6,  pp.  2205-2209;  Dec.,  1977. 

(*)  Raymond  and  Pease,  NS-2A,  No.  6,  pp. 2327-2335;  Dec.,  1977  . 
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prbcbdino  pack  buik-not  nu® 


Developing  Diagnostic/Test  Support  Techniques 


RANGE  Of  MEASURED  THHFSHOl  D UPSE  T 
LEVELS  FOR  NARROW  PULSES  l20-100nS) 
RANGE  Of  MEASURED  THRESHOLD  UPSET 
LEVELS  FOR  WIDE  PULSES  (1  S>jS) 


§ Pattern  generation  for  transient  radia 
tion  tests 


CONVENTIONAL  I*L 


• Test  elements,  coupons,  wafers 


RCA 

• FLIP  FLOPS 
NORTHROP 

• FLIP  FLOPS.  SHIFT  REGISTER 


§ Laser  excitation  as  photoresponse 
diagnostic 


G E 

• FLIP  FLOPS 


• Hardness  assurance  techniques  - critical 
tolerance  factor. 


TEXAS  INSTRUMENTS 
• SBP0400 


ADVANCE  OI2L 


Recommendations 


TEXAS  INSTRUMENTS 
• ION  IMPLANTED  ROM 


SAMSO/DoD  involvement  in  VLSI  evolution 
including  n-MOS 


HARRIS 

• SUBTRATE  FED  SHIFT 
REGISTER 

HUGHES 

• UP  DIFFUSED  FREOUENCY 
DIVIDER 


CMOS/SOS 


10'  Ttr  10y  1 

PEAK  DOSE  RATE.  X.  rail  (Si)/*ec 


Simula • y of  Integrated  Injection  logic  Transient 
Photoi  espouse . 


Combined  bipolar  analog/digital  I L 

Develop  analytical  methodologies  in 
support  of  experimental  evaluation  of 
radiation  susceptibility 


Unique  LSI  Testing  Considerations 


• Complete  characterization 


• Worst-case  electrical  bias 


• Worst-case  logic  state 


New  pattern  sensitivities 


Develop  experimental  methodologies  in 
support  of  manufacturer  and  user  hard 
ness  assurance 


Random"  critical  cell  failure 


Dependence  on  electrical  bias  during 
radiation  exposure:  long-term  ionization 
effects 


• Test  elements 


Dependence  on  electrical  bias  during 
radiation  exposure:  power  supply 
photocurrent/radi a tion- induced  latch-up 


• Diagnostic  techniques 


• Non-destructive  margin  evaluations 

Require  implementation  and  evaluation  of 
test  and  analysis  techniques  on  hardened 
arrays  under  development  (i.e., 
practice). 


• Short-term  annealing  effects 


Time  urgency  of  post-irradiation 
performance  characterization 


• Dependence  on  observable  logic  states 
during  radiation  exposure. 


2 x Multiplier  Statistical  Transient  Output  Photoresponse 


Observation  of  transient  output  photo- 
response FOR  VARIED  INPUT  CONDITIONS 
DEF I N I NO  THE  SAME  STATE  ON  A GIVEN 
OUTPUT  TERMINAL. 


Constant  radiation  exposure 


HIGH  RELIABILITY  APPLICATIONS 
OF  CUSTOM  LSI 

John  D.  Heightley 
Mgr,  Integrated  Circuit  Design 
Sandia 

The  primary  driving  forces  for  the  use  of  custom 
LSI  in  systems  are  size,  reliability,  reduced  power 
dissipation,  and  cost.  Applications  requiring  very 
high  reliability  have  significant  attributes  that 
can  dilute  the  advantages  of  custom  LSI.  It  is 
important  to  recognize  these  attributes  and  mini- 
mize their  effect.  Some  of  the  most  important  of 
these  attributes  are: 

• Production  volume  is  typically  small. 

• Design  costs  are  high. 

• Adequate  testing  is  difficult  to  develop. 

• It  is  difficult  to  establish  confidence 
in  the  reliability  of  a design  because  of 
small  production  volume. 

The  net  results  of  these  attributes  is  that  it  is 
very  difficult  to  interest  any  supplier  in  develop- 
ing custom  LSI  circuits  except  at  prohibitive 
prices.  The  primary  reason  for  this  is  that  one 
of  the  LSI  suppliers  most  valuable  resources  is  his 
design  and  engineering  talent  and  he  must  get  a 
high  return  on  every  investment  of  this  resource. 

The  approach  that  Sandia  Laboratories  has  taken  to 
the  development  of  custom  LSI  in  order  to  minimize 
the  effect  of  the  above  attributes  has  the  follow- 
ing critical  elements: 

• All  design  is  done  in-house  through  mask 
generation. 

• Standard,  well  characterized,  building 
block  cells  are  used  in  the  design. 

• Heavy  use  of  computer  aids  is  made 
throughout  the  design  to  minimize  cost 
and  maximize  confidence  in  the  design. 

• Prototype  devices  are  produced  in  an 
in-house  processing  facility  using  a 
process  available  at  several  production 
suppliers  in  the  1C  industry. 

The  use  of  standard  cells  in  this  approach  is 
critical  to  achieving  confidence  in  the  reliability 
of  an  LSI  design  when  the  production  quantities  are 
very  small.  The  cells  are  thoroughly  analyzed, 
characterized,  and  the  reliability  assessed  before 
they  are  used. 


Reliability  Advantages  of  Custom  LSI 

0 

rewer  interconnects,  packages,  and 
wire  bonds 

0 

Less  oower  dissipation  and  heat 

0 

Better  inherent  wafer  fabrication 
process  control  required  for  LSI 

Potential 

Reliability  Disadvantages  of  Custom  LSI 

0 

Difficult  to  test  adequately 

0 

Difficult  to  apply  stress  to  all  devices 

0 

Small  production  volume  little 
"learning"  nossible 

0 

Die  visual  is  difficult  to  perform 

Test  Sequence  Generation  - Present  Capability 

1) 

Manual  test  vector  generation 

2) 

Verification  by  simulation 

3) 

States  applied  and  gate  activity 
statistics 

4) 

Selectable  fault  ootions 

Test  Sequence  Generation  Future  Capability 

1) 

Automatic  and  interactive  test  vector 
generation 

2) 

Comolete  circuit  exercising 

3) 

Irredundant  dominant  set  of  faults 

4) 

Control  1 abi 1 i ty/observabi 1 i ty  measures 
for  each  node 

5) 

Stuck  fault  coverage  us'ng  parallel 
simulation 

6) 

Functional  level  modeling 

7) 

Circuit  layout  and  physical  defects 

8) 

Fault  dictionary 

Primary  Wafer-Related  MOS  Defects  and  Screens 

Defect  Screen 

Oxide  contamination  High  temperature  - bias 
stress 


By  doing  the  design  and  prototype  production 
in-house,  the  development  costs  are  kept  reasonably 
low  making  the  use  of  custom  LSI  in  exploratory 
systems  viable.  In  addition,  the  knowledge 
obtained  from  doing  the  development  in-house  is 
invaluable  since  reliability  is  treated  as  a criti- 
cal requirement  during  development.  It  Is  not 
always  possible  to  have  the  same  emphasis  in  the 
very  cost  sensitive  atmosphere  at  IC  suppliers. 


Oxide  oin  holes 


Metal  1 ization 


Static 

Dynamic 

High  voltaqe  stress  - 
dynamic 

Sem  wafer  certification 
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STANDARD  PART  DIGITAL  SYSTEM  DESIGN 


STANDARD  CELL  LSI  DESiGN 


Standard  Cell  Library 

• Circuit  designed  with  extensive  analysis 

• Layout  complete  with  extensive  checking 
for  design  rule  violations 

• Test  chip  fabricated  and  each  cell 
characterized  and  reliability  assess- 
ment performed 

• Cell  certified  for  inclusion  in  library 

• Computer  generated  layout  using  cells 

from  library  to  assure  adherence  to 
design  rules  and  interconnect 

• Analysis  performed  on  critical  paths 
using  layout  parameters 

Advantages  of  Leadless  Hermetic  Packages 

• Size 

• No  leads  to  corrode,  crack,  etc. 

• Better  temperature  cycling  character- 

istics when  on  thick  film  substrates 

• Rugged 


DESIGN  TOOLS  FOR  CUSTOM  LSI 

Jack  Hilibrand 
Staff  Technical  Advisor 
RCA-Gov't  Sys.  Div. 


Key  Words : Custom  LSI,  Circuit  Design,  Design 
Tools,  CMOS  LSI 


Abstract 


The  development  of  reliable  and  cost  effective 
Custom  LSI  Arrays  has  been  made  possible  by  the  use 
of  highly  disciplined  design  procedures  along  with 
computer-aided  design  tools. 


There  are  three  classes  of  custom  LSI  designs.  The 
most  familiar  is  the  carefully  "hand-crafted" 
design  where  an  experienced  semiconductor  circuit 
designer  converts  a complex  logic  design  into  a 
layout  of  individual  transistors  which  are  tightly 
packed  on  the  chip.  Here  is  an  example  of  a 
microprocessor,  RCA's  CDP  1802,  (Figure  3)  which 
was  designed  that  way.  A second  approach  is  to 
design  an  array  of  individual  identical  transistors 
and  to  provide  pathways  by  which  this  array  can  be 
customized  using  only  the  metal  interconnect  level 
(Figure  4).  This  is  the  Universal  Array  anproach. 
Finally,  a third  approach  is  to  design  a library 
of  multitransistor  "standard  cells"  which  perform 
each  of  the  necessary  elementary  logic  functions 
(Figure  5).  A computer  program  is  used  to  place 
the  cells  and  route  the  interconnection  network  in 
an  optimum  way.  These  latter  two  approaches  are 
the  central  topics  of  this  paper. 


Introduction 


The  use  of  custom  LSI  parts  involves  the  replace- 
ment of  off-the-shelf  SSI  and  MSI  digital  parts  by 
a smaller  number  of  more  complex  devices  especially 
designed  for  the  application  (Figures  1 and  2). 

The  conversion  to  custom  LSI  significantly  reduces 
the  number  of  parts  and  pins,  and  the  number  and 
complexity  of  PC  boards  so  that  there  is  a signifi- 
cant increase  in  reliability  and  a significant 
reduction  in  system  fabrication  cost.  In  addition, 
moving  more  of  the  interconnection  network  onto  the 
chip  improves  system  performance.  Also,  the 
reduced  size  and  weight,  and  greater  ease  of 
shielding  against  radiation  are  significant  advan- 
tages for  space  applications.  Finally  the  use  of 
subsystem  scale  logic  elements  permits  easier 
diagnosis  and  part  replacement.  It  is  the  use  of 
custom  LSI  parts  in  watches,  calculators,  games, 
instruments,  etc.,  which  has  transformed  the 
American  electronics  scene. 


Figure  2.  Comparison  of  LSI  Device  to  the  SSI 
Breadboard  (Back) 

Optimally  designed  digital  systems  of  the  future 
are  likely  to  consist  of  microprocessor  and  memory 
devices  surrounded  by  a small  number  of  Comdex 
custom  LSI  parts  which  specialize  the  specific 
function  being  performed.  This  discussion  will  be 
focussed  on  complementary  MOS  technology  which  is 
well  suited  to  such  use  by  the  low  power  dissipa- 
tion per  gate  and  the  ease  of  design  implementation 


Figure  1.  Comparison  of  LSI  Device  to  the  SSI 
Breadboard  (Front) 


The  basic  problems  with  the  hand-crafted  approach 
are  the  cost  of  this  procedure  for  a complex  LSI 
chip  and  the  numerous  opportunities  for  minor,  but 
fatal  errors  to  creep  into  the  layout.  Hand- 
crafting results  in  the  densest  layouts  and  is 
widely  used  for  memories  where  the  primary  focus 
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is  on  absolute  minimization  of  the  size  of  the 
memory  cell  which  is  then  replicated  four,  sixteen, 
or  sixty-five  thousand  times.  The  designer's 
effort  to  save  a square  micron  of  cell  area  is  well 
repaid  in  terms  of  a smaller  chip  and  improved 
yield  giving  more  net  good  chips  per  wafer,  the 
ultimate  measure  of  cost  in  the  microelectronics 
world.  The  CMOS  microprocessor  uses  seven  square 
mil/device  and  RCA's  1024  bit  CMOS  static  RAM  uses 
three  square  mils/device.  These  are  benchmarks  for 
the  hand-crafted  design  approach. 


Universal  Array  Custom  LSI 


The  Universal  Array  aoproach  is  at  the  other 
extreme  in  minimizing  design  costs  while  accepting 
significantly  poorer  oackinq  density.  Families  of 
Universal  Arrays  (Fiqure  6)  are  available  in  each 
of  the  CMOS  logic  technologies.  The  device  density 
is  half  an  order  of  magnitude  poorer  than  in  the 
handcrafted  desiqn.  The  most  complex  phase  of 
custom  LSI  design  using  the  Universal  Array,  is  the 
interconnection  of  the  devices.  This  step  is 
simplified  by  the  use  of  Dredesigned  appliques  for 
each  of  the  commonly  used  logic  functions  in  which 
the  size  of  the  logic  function  is  minimized  (Fig- 
ure 7).  These  appliques  can  be  applied  to  a 500 
scale  mylar  film  version  of  the  Universal  Array  and 
the  interconnection  pattern  can  then  be  laid  out  by 
hand.  The  interco.nection  matrix  is  then  digitized 
and  fed  into  a computer  which  generates  a magnetic 
tape  to  drive  a pattern  qenerator  and  create  the 
metallization  mask  level.  Since  wafers  can  be 
stockpiled  fully  metallized  at  the  end  of  the 
wafer  fabrication  line,  it  is  possible  to  get 
finished  devices  within  a few  weeks  after  the 
interconnection  pattern  is  defined.  Similarly, 
loqic  errors  can  be  corrected  and  imorovements  in 
the  system  desiqn  can  be  implemented  by  relatively 
quick  modifications  of  the  sinqle  mask  level.  In 
fact  the  advantage  of  using  a Universal  Array 
approach  for  early  system  desiqn  stems  from  the  low 
cost  and  quick  turnaround  possible  when  making 
minor  changes. 


Figure  3.  CDP  1802  CMOS  Microprocessor 
5500  Devices;  7.6  Square  Mils/Device 


Figure  4.  Universal  Array  Synch  Generator  Chip 


Figure  6.  Universal  Array  Families 


The  Standard  Cell  apDroach,  also  called  APAR 
(Automatic  Placement  And  Routing),  provides  a more 
highly  automated  scheme  for  custom  LSI  design.  A 
library  of  thirty  to  fifty  logic  elements  (Fig- 
ure 8)  is  created,  each  of  which  Is  of  constant 


Figure  5.  Standard  Cell  Arithmetic  Logic  Unit 


* * 


BINARY  1'IVIULK  1 ! 2) 


Figure  7.  Binary  Divider  Logic  Cell  Applique 


designers  are  not  generally  semiconductor  experts. 
They  need  access  to  an  LSI  design  approach  which 
will  not  only  be  cost  effective  (by  minimizing 
front-end  costs)  but  one  which  will  not  require 
them  to  learn  the  complexities  of  each  new  semi- 
conductor technology  as  it  comes  along,  and  one 
which  provides  a high  degree  of  assurance  that  the 
design  cycle  will  be  timely  and  the  resulting  chip 
will  be  reliable.  The  design  cycle  (Figure  11) 
involves  interaction  between  the  RCA  system  activi- 
ties and  the  RCA  Solid  State  Technology  Center 
where  the  semiconductor  masking,  wafer  fabrication, 
and  parts  assembly  and  test  are  implemented. 

The  Solid  State  Technology  Center,  a central  cor- 
porate facility  (Figure  12)  was  established  seven 
years  ago  to  provide  a pilot  line  for  advanced 
technologies  where  small  quantities  of  mil-spec 
qualifiable  parts  could  be  gotten  on  a cost  effec- 
tive, quick-turnaround  basis.  The  standard  cell 
library  (Figure  13)  is  reviewed  by  a corporate 
group  for  conformance  to  the  design  rules.  Each 
cell  is  incorporated  into  a test  vehicle  chip  (Fig- 
ure 14)  in  which  it  can  be  individually  burned-in 
and  life  tested.  When  this  process  is  complete, 
the  standard  cell  library  is  approved  for  use  in 
high  reliability  applications  and  the  cells  are 
secured  against  all  unauthorized  changes.  A similar 
procedure  is  used  to  formally  adopt  the  Universal 
Array  for  high  reliability  applications.  The  cells 
and  UA  elements  are  characterized  and  User  Manuals 
are  issued  to  enable  designers  to  access  these 
tools.  These  procedures  ensure  that  the  basic 
building  blocks  for  custom  LSI  designs  are  well 
understood  and  reliable  so  that  equipment  designers 
can  use  them  with  confidence. 


height  and  obeys  specific  rules  on  where  the  power 
and  ground  leads  are  tapped  and  where  the  signal 
inputs  and  outputs  are  available.  These  cells  are 
laid  out  by  experts  to  meet  the  design  rules  of  the 
technology  being  used  while  offering  minimum  size 
and  good  performance.  Interior  devices  inside  the 
cell  can  be  minimized  in  size  while  those  which 
interface  with  the  interconnection  network  are 
sized  to  drive  on-chip  wiring  loads  and  those  which 
drive  the  output  pins  of  the  device  are  made  still 
larger  to  drive  the  board  wiring.  The  cell  layouts 
are  then  stored  in  computer  memory.  Computer  pro- 
grams have  been  developed  which  take  the  intercon- 
nection net  list  and  optimally  place  the  cells  and 
route  the  interconnection  wiring.  The  design  engi- 
neer, who  need  not  be  expert  in  semiconductor 
design,  examines  a checkplot  (Figure  9)  of  the  lay- 
out which  he  can  then  modify  slightly,  if  he  wishes, 
to  improve  the  packing  density. 

Which  Approach  to  Use 

These  three  approaches  to  custom  LSI  cover  the 
spectrum  of  customer  requirements  (Figure  10).  For 
low  volume  systems  of  five  hundred  gate  complexity 
or  less,  the  Universal  Array  is  a cost  effective 
approach.  For  high  volume  systems  where  getting  to 
the  absolute  minimum  chip  size  is  important,  hand- 
crafted arrays  are  used.  In  the  large  middle  area, 
standard  cell  approaches  are  most  cost  effective 
and  this  is  the  technique  used  most  widely  in  the 
RCA  Government  Systems  activities. 

Design  Disciplines  for  Custom  LSI 

Let's  get  some  perspective  on  the  design  disci- 
plines for  reliable  custom  LSI.  Military  equipment 


Conclusions 

RCA  has  organized  to  make  Custom  LSI  technology 
available  for  a variety  of  systems  applications 
because  of  our  belief  that  the  use  of  custom  CMOS 
LSI  parts  (along  with  the  microprocessors  and 
memories)  is  the  most  reliable  and  cost-effective 
way  to  implement  space  systems. 

Recommendations 

In  order  to  make  the  use  of  custom  LSI  Darts  practi- 
cal: (1)  Special  attention  is  needed  to  provide 
for  custom  LSI  part  desiqn  early  in  the  prototype 
and  testinq  phase  of  programs  so  that  the  life  cycle 
cost  benefits  from  the  use  of  LSI  parts  can  be 
achieved  during  the  production  phase  of  the  program. 
(2)  Custom  LSI  parts  should  be  qualified  on  a fam- 
ily basis  through  testinq  of  standard  cell  libraries 
and  Universal  Array  Vehicles  rather  than  by  fully 
qualifying  each  custom  part  individually. 
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Figure  9.  Checkplot  of  a Standard  Cell  Array 
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Figure  11.  Design  and  Fabrication  of  Standard 
Cell  Custom  LSI  Arrays 


Figure  14.  Test  Vehicle  Chin 


integrated  circuit  line.  Since  1971,  he  has  been 
responsible  for  planning  and  coordinating  mono- 
lithic LSI  development  and  hybrid  circuit  activi- 
ties for  the  Business  Units  of  the  RCA  Government 
Systems  Division. 


Figure  12.  RCA  Solid  State  Technology  Center 
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Figure  13.  Custom  LSI  Array  Development  with 
Existing  High-Rel  Cells  Plus  the 
Introduction  and  Approval  of  New 
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PATTERN  GENERATION  METHODS 
FOR  MICROPROCESSORS 

Lenward  E.  Holness 
Mgr.,  Evaluation  Dept. 
Hughes  Aircraft  Co. 


LEARNED  RESPONSE  METHOD 


DISADVANTAGES 


ADVANTAGES 


• MODE  RAGE  IMPLEMENTATION  EFFORT 


• NOT  LIKELY  TO  DETECT  LOGIC,  DESIGN 
OR  DOCUMENTATION  ERRORS 


• FLEXIBLE 


t Hardware  methods 


• REQUIRES  EXPENSIVE  TEST  SYSTEM 


• Software  methods 


t Functional  simulation  example 


SOFTWARE  PATTERN  GENERATION 


§ Summary 


LOGIC  SIMULATION 


Hardware  Pattern  Generation  Methods 


ADVANTAGES 


DISADVANTAGES 


• Hardware  pattern  emulation 


• WILL  THEORETICALL  Y COVER  99% 
OF  THE  DUT*  CIRCUIT  NODES? 


• DOES  NOT  PROVE  THAT  DUT  PERFORMS 
DOCUMENTED  FUNCTIONS 


• Algorithmic  pattern  generation 


• WILL  PROVE  DUT  MATCHES  CIRCUIT 
SCHEMATIC 


• REQUIRES  THE  CORRECT  CIRCUIT 
SCHEMATIC 


• CANNOT  BE  USED  TO  GENERATE 
APPLICATION  ORIENTED  PATTERN 


• WILL  USUALLY  PRODUCE  SHOHT 
EFFICIENT  PATTERN 


HARDWARE  PATTERN 
GENERATION  METHODS 


• SIGNIFICANT  IMPLEMENTATION  EFFORT 


DISADVANTAGES 


• REQUIRES  BOTH  HARDWARE  AND  SOFTWARE 
EMULATION 


• EFFICIENT.  I.E  . REOUIRES  LITTLE  TEST 
SYSTEM  OVERHEAD 


SOFTWARE  PATTERN  GENERATION 

FUNCTIONAL  SIMULATION 


• EMULATOR  MUST  BE  FASTER  THAN  THE  TEST 
DEVICE 


• DIFFICULT  TO  GENERATE  NON  ALGORITHMIC 
PATTERNS 


• MUST  BE  IMPLEMENTED  IN  LOW  LEVEL 
LANGUAGE 


• DIFFICULT  TO  CHECK  PROGRAM 


COMPARISON  METHOD 


THE  PRODUCTION  CYCLE 


Software  Pattern  Generation  Methods 


• Manual 


• Logic  simulation 


• Functional  simulation 


ADVANTAGES 

. 

DISADVANTAGES 

• REAL  TIME 

• LIMITEO  TO  TIMING  ANO  SPEED  CONSTRAINTS 

OF  NCO 

• LIMITED  PATTERN  STORAGE 

• NOT  LIKELY  TO  DETECT  LOGIC.  DESIGN  OR 

DOCUMENTATION  ERRORS 

• REOUIRES  INEXPENSIVE  TESTER 

• LIMITED  FLEXIBILITY 

1 

ADVANTAGES 

DISADVANTAGES 

• CAN  GENERATE  APPLICATION  ORIENTED 
PATTERNS 

• SIGNIFICANT  IMPLEMENTATION  EFFORT 

• PROVES  DUT  PERFORMS  DOCUMENTED 
FUNCTIONS 

• CANNOT  PROVE  PATTERNS  EXERCISE 
EVERY  CIRCUIT  NODE 

• CAN  GENERATE  AOOITIONAL  PATTERNS 
WITH  MINIMAL  EFFORT 

• LIMITEO  f AULT  ISOLATION 

INFORMATION 

• PROVIDES  CLOCK  LEVEL  AND  BIT  LEVEL 
DESCRIPTION  OF  MICROPROCESSOR 
OPERATION 

DEVICE  DESCRIPTION  EXAMPLE 


FUNCTION  TABLE  FOR  54S194 
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FUNCTIONAL  SIMULATION 
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MICROPROCESSOR  SLASH  SHEET  DEVELOPMENT 


Laboratories,  NASA,  Hughes  Aerospace,  Macrodata 
Corporation,  and  Continental  Test  Laboratories. 


W.  Richard  Scott 
Supervisor 

Jet  Propulsion  Laboratory 


I was  asked  by  the  Workshop  Chairman  to  discuss  the 
preparation  of  military  specifications  for  large 
scale  integrated  circuits.  Yesterday,  Col. 
Schlosser,  SAMSO,  and  Larry  Murphy,  NASA,  discussed 
a SAMSO/NASA  "marriage"  out  of  which  would  come  a 
joint  effort  to  accelerate  the  writing  and  release 
of  Class  S military  specifications.  JPL  has  a con- 
tract with  the  NASA  Standard  Parts  Program  Office 
to  do  just  that  - write  specifications  for  the 
1802,  1822,  1852  and  1853. 

Lee  Farnham  of  GE  spoke  yesterday  of  the  evolution 
of  the  military  specifications,  starting  with  the 
JAN  and  progressing  through  the  TX,  TXV,  Class  A 
and  finally  Class  S.  He  also  imputed  to  them  the 
ability  to  provide  1,  2,  3,  5 and  10  year  parts 
respectively.  I disagree  with  those  numbers,  per 
se,  but  further  state  that  Class  S as  it  exists 
today,  is  not  adequate  for  high  reliability  LSI 
devices. 

Due  to  the  significantly  increased  complexity  of 
semiconductor  fabrication  processes  as  well  as  chip 
design,  there  exist  many  new  physical  failure 
modes.  That  coupled  with  the  immaturity  of  the 
whole  LSI  technology,  lends  the  SSI  and  MSI  tech- 
niques inadequate  for  LSI. 

Before  starting  the  draft  of  the  MIL-M- 38510/470, 
the  1802  slash  sheet,  JPL  coordinated  the  specifi- 
cation requirements  with  the  Air  Force,  Army,  Navy, 
NASA,  the  semiconductor  industry  and  both  commer- 
cial and  high  reliability  users. 

Preparing  such  a specification  begins  with  a char- 
acterization test  program.  We  started  with  collec- 
tion and  review  of  vendor  and  user  data,  construc- 
tion analysis,  and  further  discussions  with 
manufacturers  and  users.  From  there  we  developed 
a characterization  test  in  which  we  developed  a 
number  of  test  vectors.  We  then  ran  sample  devices 
through  dc,  switching,  and  functional  tests,  vary- 
ing voltage,  temperature,  and  frequency.  After 
repeating  these  tests  numberous  times  we  selected 
what  we  felt  to  be  the  worst  case  vectors  to  which 
the  device  appears  most  error  sensitive  for  a 
given  set  of  timing  conditions,  voltages,  and 
environmental  constraints.  his  was  an  iterative 
process  and  from  it  we  identified  approximately 
14,000  test  vectors.  These  become  part  of  the 
screening  and  qualification  requirements. 

The  specification  identifies  the  parameters, 
parameter  limits  and  test  conditions.  It  is 
designed  to  examine  for  pattern  sensitivities  and 
establishes  timing,  power  and  temperature  levels. 
Burn-in  circuits  and  conditions  are  defined  as  are 
visual  inspection  criteria. 

Before  completing  the  draft,  and  hoping  to  shorten 
the  usual  12  to  18  month  coordination  cycle  for 
military  specifications,  we  held  a precoordination 
conference  a month  ago.  Present  were  all  the  manu- 
facturers of  the  1802  (RCA,  Hughes,  and  Solid  State 
Scientific)  as  well  as  RADC  and  SAMSO.  Because  of 
their  work  on  LSI,  we  also  Invited  Sandia 


The  deliverables  under  this  contract  include  the 
draft  specifications  as  well  as  test  tapes  pro- 
grammed with  all  the  test  vectors  and  parameter 
measurements.  These  tapes  are  written  for  immedi- 
ate use  on  the  Tektronix  S3260.  Copies  of  these 
will  be  available  either  through  the  NASA  Standard 
Parts  Program  Office  at  MSFC  or  through  RADC. 

In  conclusion  we  feel:  1)  that  complete  charac- 
terization testing  must  be  accomplished  before  a 
specification  can  be  written;  2)  the  micro- 
processors have  several  serious  limitations  the 
most  notable  one  of  which  is  their  lack  of  test- 
ability; 3)  the  ability  to  oerform  functional  and 
switching  test  is  seriously  limited  by  most  test 
systems;  4)  no  complex  LSI  can  be  expected  to  meet 
all  the  requirements  of  MIL-M-38510,  Class  S,  as 
they  now  exist;  and,  5)  because  of  all  the  subtle 
and  not  so  subtle  problems  encountered  in  charac- 
terizing and  specifying  microprocessors,  there  is 
an  urgent  need  for  a single  clearing  house  to 
collect,  analyze,  and  disseminate  the  LSI  informa- 
tion to  al 1 agencies. 
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Precoordination  Conference 
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• Draft  MIL-M- 38510/470 

• S3260  test  program  tape 
Concl usions 
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• Device  limitations 
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During  the  past  several  years  memory  technology  has 
become  a critical  part  of  modern  space  and  military 
digital  systems.  This  is  a by-product  of  the 
microprocessor  revolution  and  the  sophistication 
required  for  onboard  computation  in  increasingly 
complex  miniaturized  systems. 

The  equivalent  gate  count  of  these  memory  devices 
is  so  large  as  to  overshadow  the  equivalent  gate 
count  in  the  remaining  digital  circuitry  and  for 
this  reason  reliability  computations  based  upon 
MIL-HDBK-217B  are  extremely  sensitive  to  improve- 
ments in  memory  reliability  technology. 

Part  screening  must  be  employed.  A hierarchy  of 
screening  level  begins  with  MIL-M-38510  and  can 
accept  lesser  parts  with  proper  screening  and 
burn-in  test  performed  by  the  device  vendor  or 
testing  laboratory.  This  allows  the  use  of  opti- 
mistic quality  factors  from  the  MIL-HDBK-217B 
reliability  model. 

Additional  cost-effective  tests  which  need  to  be 
developed  are  described  herein. 

As  new  technologies  such  as  magnetic  bubbles 
emerge  in  operational  military  systems,  new  speci- 
fications will  be  developed  to  deal  with  their 
screening  and  burn-in  requirements.  Other  tech- 
nologies which  are  basically  similar  to  existing 
MOS  and  bipolar  devices  should  require  only  minor 
modifications  to  the  existing  MIL-STD-883  format. 

Introduction 

During  the  past  several  years  memory  technology  has 
become  a critical  part  of  modern  space  and  military 
digital  systems.  This  is  a by-product  of  the 
microprocessor  revolution  and  the  sophistication 
required  for  onboard  computation  in  increasingly 
complex  miniaturized  systems.  Because  of  the 
repetitive  nature  of  memory  structures,  chip 
densities  for  memories  are  the  largest  in  the  LSI 
and  VLSI  domain.  The  I/O  problem  associated  with 
these  large  devices  is  minimized  by  binary  address 
lines  or  by  serial  access.  Thus,  the  usual  limita- 
tions in  chip  complexity  that  place  an  upward 
bound  on  single  chip  size  have  been  minimized  and 
future  growth  beyond  65  kilobits  is  expected  in 
the  immediate  future. 

In  addition,  current  and  future  systems  require 
several  thousand  words  of  processor  instructions 
stored  in  ROM,  1000  to  2000  bits  of  scratch  pad 
RAM,  possibly  a megabit  of  additional  RAM  and  up  to 
10®  bits  of  bulk  storage.  The  system  design  con- 
straints of  cost,  size,  weight,  power  and  reli- 
ability are  focusing  attention  on  solid  state 
memories  to  fill  these  needs.  The  present  state- 
of-the-art  utilizes  primarily  silicon  technology. 
However,  other  specialized  devices  (e.g.,  magnetic 
bubble,  crosstie  and  FAM0S  memories)  are  soon  to 
be  parts  of  operational  space  and  military  systems. 
(Figure  1) 


Figure  1.  Memory  Hierarchy 

The  equivalent  gate  count  of  these  memory  devices 
is  so  large  as  to  overshadow  the  equivalent  gate 
count  in  the  remaining  digital  circuitry  and  for 
this  reason  reliability  computations  based  upon 
MIL-HDBK-217B  are  extremely  sensitive  to  improve- 
ments in  memory  reliability  technology.  That 
improvement  has  three  sources: 

• Improvements  in  device  processing  and 
packaging  technology 

• Improved  screening  and  burn-in 
procedures 

• Better  design  guidelines  and  operating 
environment 

Design  Trends 

Today's  engineer  has  a choice  of  a variety  of 
memory  products  which  are  built  from  several  MOS 
and  bipolar  technologies.  Each  has  its  advantages 
and  disadvantages  and  becomes  more  or  less  com- 
petitive as  technology  develops.  In  general,  the 
choice  for  a RAM  is  made  based  upon  access  time 
requirements,  storage  capacity  and  cost.  The 
table  below  shows  general  trade-off  parameters 
utilized  in  the  selection  process.  ROMs,  PROMs, 
EAROMs  are  a bit  more  controversial  in  their 
selection  criterion.  Most  processor  instructions 
and  look-up  tables  are  stored  in  ROMs.  However, 
schedules  often  require  use  of  PROMs  to  facilitate 
last  minute  changes  in  software.  In  addition, 
avionic  prime  and  test  equipment  usually  require 
flexibility  which  can  be  best  achieved  utilizing 
PROMs.  EAROMs  are  in  general  a poor  choice  for 
program  storage  and  scratch  pad  applications.  How- 
ever, they  provide  an  attractive  alternative  to 
magnetic  memories  or  a battery  backup  for  non- 
volatile storage  data.  (Figure  2) 
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Figure  2.  Memory  Selection  Criteria 


At  FSEC,  our  application  of  memories  follows  the 
general  guidelines  described  above  on  space  and 
avionic  programs.  Both  prime  and  test  equipment 
are  involved  for  each  program. 


For  any  given  device  type  several  design  guidelines 
are  applied  to  maximize  the  reliability  of  the 
device. 

• Temperature  - Temperature  is  a critical 
factor  in  both  failure  rate  and  vola- 
tility. It  is  especially  critical  since 
the  high  density  of  gates  produces  a 
large  power  dissipation  even  with  MOS 
devices.  Dissipation  values  of  1/2  watt 
and  more  are  common,  and  memory  hybrids 
or  circuit  boards  can  typically  dissipate 
5-10  watts.  Recent  military  guidelines, 
especially  those  from  the  Navy,  encourage 
junction  temperatures  of  1 1 OOC  or  less 
even  in  a 71°C  (MIL-E-5400  Class  II) 
environment.  Board  designs  can  utilize 
laminated  metal  or  metal  core  construc- 
tion to  improve  heat  flow  to  the  board 
edges  where  conduction  or  forced  air 
cooling  is  provided.  Techniques  utiliz- 
ing ceramic  board  or  ceramic  bar  tech- 
nology shows  promise  for  improved  heat 
flow.  A selective  “power-down"  mode  can 
also  be  employed  to  reduce  dissipation 
during  idle  times  under  the  control  of 
the  processor. 

• Timing  Criticality  - While  not  as  impor- 
tant as  temperature,  timing  parameters 
are  both  related  to  internal  device 
operation  and  the  device-circuit  inter- 
face. They  are  a function  of  many  param- 
eters which  include  device  processing 
effects  and  the  device  operating  envi- 
ronment. Breadboard  tests  demonstrate 
design  adequacy  for  a nominal  set  of 
values,  but  proper  worse  case  and  sneak 
circuit  analyses  must  be  performed  to 
insure  reliable  operation  for  the  dura- 
tion of  the  program.  In  some  cases, 
device  screening  has  shown  that  a fail 
condition  can  exist  within  an  otherwise 
acceptable  range  in  the  variation  of  a 
parameter  between  minimum  and  maximum 
values. 

• Shielding  - This  factor  is  particularly 
important  where  ultraviolet  erasible 
PROMs  are  utilized.  Foil  tape  is  used 
to  guard  against  unwanted  ultraviolet 
leakage  into  the  device  window.  In 
addition,  many  common  MOS  devices,  espe- 
cially PROMs  and  EAROMs,  are  susceptible 
to  radiation  and  until  hardened  devices 
are  developed,  suitable  shielding  must 
be  provided. 

• Use  of  Redundancy  - A common  way  to 
improve  the  system  reliability  is  through 
the  use  of  redundancy.  Many  military 
systems  contain  complex  buf 1 t-in-test 
routines  which  verify  the  integrity  of 
both  RAM  and  ROM  devices.  Test  failures 
will  automatically  trigger  the  switch- 
over to  backup  memories  or  an  entire 
backup  processor.  Degraded  mode  assess- 
ment is  another  technique  to  create  fail 
soft  conditions  when  portions  of  the 

RAM  or  ROM  fail. 


Device  Defects 

While  many  of  these  devices  are  reasonably  new  on 
the  scene,  failure  histories  are  being  developed 
by  Rome  Air  Development  Center  and  industry.  The 
table  below  summarizes  some  preliminary  analyses 
conducted  by  RADC  on  LSI  failures  for  various 
technologies.  While  all  of  these  devices  are  not 
memories,  the  data  is  representative  of  expecta- 
tions for  memory  devices.  The  leading  defect 
varies  with  device  technology,  however,  contamina- 
tion is  significant  in  all  cases  and  other  defects 
could  stem  from  registration  problems  with  these 
large  structures.  The  former  factor  tends  to 
stress  the  importance  of  precap  visual,  SEM,  and 
leak  testing  in  screening.  The  RADC  study  also 
tabulated  packaging  failure  data.  These  results 
are  typical  of  the  extensive  work  that  is  con- 
tinuing at  the  RADC  Reliability  Analysis  Center. 
(Figure  3 and  Figure  4) 
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Figure  3.  Die  Related  Failure 
Modes  Defect  Classification 


Figure  4.  Die  Related  Failure 
Modes:  LSI 


Part  Screening 

All  parts  that  are  utilized  in  space  and  military 
systems  receive  a series  of  screening  and  burn-in 
tests  which  are  in  general  in  accordance  with 
MIL-STD-883  as  amended.  In  general,  a hierarchy 
is  established  which  considers  parts  in  the  follow- 
ing order  of  preference. 

1)  MIL-M-38510 

2)  MIL-STD-883  (Level  S or  B) 

3)  Vendor  equivalent  to  MIL-STD-883B  with 
additional  screening. 
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4)  MIL  temperature  range  part  with 

additional  screening.  If  possible,  this 
will  include  precap  visual  performed  by 
the  vendor. 

Utilizing  these  guidelines  reliability  factors  are 
developed  consistent  with  MIL-HDBK-217B.  The 
quality  factor  from  that  model  is  very  sensitive  to 
the  maturity  of  the  part  and  the  adequacy  of  the 
screening  and  burn-in  process.  In  general,  when 
the  above  steps  are  taken  ng  factors  (which  range 
from  2.5  to  150)  can  be  maintained  in  the  2.5  to 
5 range.  Several  testing  laboratories  have  emerged 
which  perform  sophisticated  screening  and  rescreen- 
ing of  these  sophisticated  components.  The  table 
below  illustrates  the  categories  of  tests  that 
are  performed  on  each  generic  type  of  memory. 
(Figure  5) 
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Figure  5.  Testing  Techniques  by  Device 
Type  (RAM,  ROM,  EAROM) 

One  of  the  most  important  tests  involves  the  writ- 
ing and  reading  of  memory  cells  in  the  array. 

This  test  is  critical  since  it  checks  the  basic 
functioning  of  the  component.  It  also  is  extremely 
time  consuming  for  large  arrays  with  relatively 
slow  access  times.  A worse  case  pattern  would 
require  2N?  tests  where  N is  the  number  of  memory 
cells.  For  large  MOS  memories  this  testing  could 
take  several  minutes  which  can  result  in  long  and 
expensive  screening  tests.  Other  patterns  have 
been  developed  which  are  shown  in  the  next  table. 
However,  these  require  an  understanding  of  the 
detailed  digital  design  of  the  memory  in  order  to 
insure  that  an  adequate  test  is  performed.  This 
may  be  derived  either  analytically  or  by  experi- 
mentally testing  a group  of  samples  with  several 
patterns  to  determine  which  one  discovers  the 
largest  number  of  failures.  (Figure  6) 
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Figure  6. 

Pattern  Library 

addition  to 

the  more 

common  tests  which 

dictated  by  MIL-STD-883,  there  are  other  tests 
or  other  criterion  which  may  be  addressed  to  deter- 
mine their  effectlvity  In  Improving  the  failure 
rate  of  the  selected  device. 

• Volatility  - Volatility  is  a critical 
parameter  for  PROMs  and  EAROMs.  In 


general,  temperature  is  involved  as  a 
discriminant  even  with  fusible  link 
PROMs. 

• Thermal  Resistance  - Since  most  memory 
devices  dissipate  substantial  amounts  of 
heat,  a proper  bond  between  the  die  and 
the  package  is  critical  for  uniform  heat 
transfer.  The  presence  of  significant 
voids  can  cause  device  overheating  as 
well  as  thermal  gradients  and  stress  on 
the  chip.  This  parameter  could  be 
measured  either  by  examining  an  indi- 
vidual P-N  junction  or  possibly  by 
measuring  the  incremental  change  in  sup- 
ply current  with  increased  dissipation. 
The  thermal  resistance  also  includes 
package  thermal  spreading  and  conductive 
pads  to  the  circuit  board. 

• Current  Transients  - Switching  phenomenon 
within  the  device  can  result  in  large 
current  spikes  which  can  cause  problems 
within  the  chip  or  at  the  system  level. 
Suitable  criteria  would  determine  accept- 
able limits  for  such  current  transients. 

In  general  , the  total  screening  and  burn-in 
requirement  raises  the  basic  commercial  part  price 
by  at  least  a factor  of  three  (exclusive  of  lot 
charges)  but  the  benefit  in  improved  failure  rate 
is  many  times  that  amount.  As  new  technologies 
such  as  magnetic  bubbles  emerge  in  operational 
military  systems,  new  specifications  will  be 
developed  to  deal  with  their  screening  and  burn-in 
requirements.  Other  technologies  which  are 
basically  similar  to  existing  MOS  and  bipolar 
devices  should  require  only  minor  modifications  to 
the  existing  MIL-STD-883  format. 

As  a designer  and  manufacturer  of  digital  systems, 
we  are  faced  today  with  a need  for  large  and  reli- 
able memories  since  they  often  dominate  the  MTBF 
of  the  overall  processing  system.  The  solutions 
to  this  problem  are  addressed  as  a three  pronged 
approach  that  utilizes  proper  design  guidelines, 
improvements  in  IC  manufacturer's  technologies 
and  processes  and  cost  effective  but  comprehensive 
screening  and  burn-in  by  the  device  supplier  or  a 
screenino  laboratory.  Since  LSI  technology  is 
extremely  dynamic  today,  all  three  factors  must  be 
addressed  and  updated  continually  to  be  sensitive 
to  the  needs  of  mission  assurance.  However,  test- 
ing must  be  truly  cost  effective  in  this  era  of 
declining  budgets.  Designed  to  Cost,  and  total  Life 
Cycle  Cost. 

When  these  factors  are  successful,  qualification, 
burn-in  and  acceptance  testing  at  the  system  level 
are  cost  effective  and  lead  to  a successful  pro- 
gram and  mission. 
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CLASS  S LSI  PLANS 

Alan  J.  Car lan 
The  Aerospace  Corporation 


Abstract 

The  complexity  of  LSI  circuits  and  inability  to 
perform  functional  testing  under  all  rated  operat- 
ing conditions,  create  new  problems  in  reliability 
assessment,  qualification  and  development  of 
detailed  specifications  for  high  reliability  LSICS 
for  use  in  space  and  launch  vehicle  applications. 

At  present,  military  microcircuit  specifications 
for  Class  S do  not  include  adequate  requirements 
for  qualification  and  acceptance  of  complex  LSIC's. 
This  paper  will  describe  some  of  the  planned 
approaches  toward  extending  existing  requirements 
as  well  as  development  of  new  military  requirements 
for  use  of  LSIC's  in  space  and  launch  vehicle 
applications. 

Introduction 

• Increased  usage  of  large  scale  integrated 
circuits  in  space,  satellite  and  launch 
vehicle  systems  is  anticipated.  To 
assure  mission  success,  these  systems 
require  the  use  of  Class  S reliability 
parts. 

• However,  reliability  assessment  and 
qualification  of  LSICs  are  difficult  due 
to  their  complexity  and  inability  to  per- 
form complete  functional  testing  under 
all  rated  operating  conditions. 

• This  paper  will  briefly  review  the 
present  system  for  qualification  of 
microcircuits,  some  considerations  and 
concerns  regarding  LSICs  and  the  planned 
approaches  toward  extending  the  Class  S 
requirements  to  Include  LSICs. 

The  QPL' System  for  Class  S Microcircuits 

• Manufacturers  obtain  Class  S line 
certification. 

• A detailed  specification  (M38510  "slash" 
sheet)  is  prepared  and  issued. 

• A manufacturer  who  desires  that  a given 
part  be  placed  on  the  QPL  must  contact 
the  qualifying  activity  and  submit  the 
data  required  by  MIL-M- 385100.  The 
qualifying  activity  then  Issues  an 
authorization  to  test. 

e For  Class  S,  two  device  lots  are  sub- 
jected to  the  qualification  testing  as 
required  by  Method  5005  of  MIL-STD-883B 
and  the  applicable  detailed  specifica- 
tion. The  manufacturer  must  certify  that 
the  material  and  physical  dimensions  of 
the  microcircuits  comply  with  the  general 
and  detailed  specifications. 

• After  satisfactory  completion  of  the 
tests  and  approval  by  the  qualifying 
activity,  the  parts  are  placed  on  the 
QPL. 


Considerations  and  Concerns  Regarding  Use  of  LSICs 
in  Space,  Satellite  and  Launch  Systems 

• Potential  advantages 

• Lower  system  cost 

• Improved  system  performance 

• Facilitates  expanded  system 
capability 

• Reduces  power  consumption 

• Reduces  weight 

• Fewer  interconnections  may  improve 
system  reliability 

• Use  of  radiation  shielding  is 
simplified  due  to  small  size  of 
LSICs 

• Potential  disadvantages 

• Use  and  reliability  experience  in 
space  systems  is  almost  nonexistent. 

• A realistic  base  of  reliability 
data  is  difficult  to  obtain  due  to 
newness  of  processes  and  device 
designs,  for  example:  several  "A" 
revisions  are  appearing  for  micro- 
processors, RAMs  are  undergoing  one 
or  more  "shrinks." 

• Rapid  obsolescence  of  designs  may 
make  replacement  parts  difficult  to 
obtain. 

• It  is  impossible  to  completely  test 
LSICs  under  all  rated  operating  con- 
ditions and  inputs. 

• For  custom  LSICs;  design,  qualifica- 
tion and  delivery  of  acceptable 
parts  may  be  costly  and  time 
consuming. 

• Specific  reliability  concerns 

• Many  LSICs  are  new  or  not  widely 
used,  thus  evaluation  and  test 
history  is  very  limited  and  unproven 
parts  may  be  designed  into  systems. 

• Inability  to  perform  complete  func- 
tional testing  devices  may  be 
accepted  with  unknown  failure  modes. 
Circuit  sensitivities  may  not  be 
found,  these  may  occur  as  a result 
of  a unique  combination  of  voltages, 
combinatorial  patterns,  noise,  tem- 
perature effects,  etc. 

• Accelerated  functional  testing  in 
the  traditional  sense  is  difficult 
or  impossible  to  perform  e.g.  many 
LSICs  do  not  function  at  200°C. 


• LSICs  have  thousands  of  active 
devices.  Interconnects,  contacts 
crossovers,  as  well  as  multiple 
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conductor  layers  separated  by  thin 
dielectrics.  A failure  in  any  of 
these  can  cause  failure  of  the 
entire  circuit. 

• Due  to  the  complexity  of  the  ISICs, 
workmanship  and  other  defects  are 
difficult  or  impossible  to  find  dur- 
ing device  screening.  Defects  in, 
for  example,  thin  dielectric  may 
fail  in  a time  dependent  mode,  these 
weak  dielectrics  may  not  be  accessed 
during  functional  testing  or 
burn-in. 

• Physical  dimensions  of  the  compo- 
nents of  an  LSIC  are  getting 
smaller:  for  example: 

5 years  ago,  metal  line  width 
were  of  the  order  of  10  urn. 
LSICs  on  the  drawing  boards 
currently  have  dimensions  in 
the  1 to  3 urn  range. 

• The  potential  for  defects  and  the 
inability  to  perform  visual  inspec- 
tions on  these  components  with  small 
dimensions  coupled  with  the  increas- 
ing use  of  multiple  dielectric  and 
conducting  layers  is  enormous. 

• The  very  high  density  LSICs  may  be 
more  susceptible  to  ESD  damage  due 
to  use  of  thinner  dielectrics  and/or 
smaller  dimensioned  components  (e.g. 
junctions).  These  may  not  be  able 
to  dissipate  transient  energies 
without  sustaining  damage. 

• Custom  LSICs  have  all  of  the  con- 
cerns of  standard  LSICs  but  lack 
extensive  test,  reliability  and 
functional  performance  history. 

Each  custom  LSIC  is  a unique  system. 

Perhaps  this  workshop  can  provide  answers  to  the 
following  questions  relating  to  custom  LSICs: 

• Should  custom  LSICs  be  included  in  the 

military  QPL  system? 

• If  this  is  desirable,  how  can  it  best  be 

accomplished? 

Plans  for  Extending  Class  S LSIC  Requirements 

• Use  of  systems  approach 

• Design  reviews 

• Fault  analysis 

• Sensitivity  analysis 

• Testability  analysis 

• Requirements  for  "built-in"  testability 

• Establishment  of  a testability 
"figure  of  merit"  to  be  used  as  a 
measure  for  accepting  new  LSIC 
designs. 


t Extension  of  the  concept  of  line 

certification  to  that  of  "technology 
certification"  which  would  include  not 
only  the  present  requirements  such  as: 

• Product  assurance  program 

• Manufacturing  baseline 

• Manufacturing  change  control 
But  would  add  evaluation  of: 

• Design  rules 

• Cell  libraries 

• CAD  and  design  philosophy 

• Expand  the  existing  concept  of  wafer 
lot  acceptance  to  include  use  of  reli- 
ability test  devices  (RTDs).  These 
would  be  designed  for  use  with  each 
technology  and  mask  sequence  of  interest. 
The  RTDs  would  be  "dropped-into"  several 
sites  on  each  device  wafer.  The  RTD 
test  patterns  would  include  the  follow- 
ing (these  are  also  listed  in  the  draft 
of  MIL-STD-1 547 ) . 

• Basic  electrical  parameters  of  the 
components  of  the  LSIC. 

• Dielectric  test  structures  for 
electrical  dielectric  defect  tests 
(pinholes)  and  dielectric  breakdown 
vol tage. 

s Thermal  stability  (C-V  stress  test 

patterns)  - tests  for  contaminated 
oxides. 

• Junction  and  device  breakdown 
voltage. 

• Metal  continuity  and  stability 
(absence  of  electromigration)  over 
many  circuit  surface  features 
(steps). 

• Metal  to  semiconductor  and  poly- 
silicon contacts  for  stability 
under  current  temperature  stresses. 

• Sheet  resistance  of  diffused  or 
Implanted  regions, 

• Critical  circuit  cells. 

• Line  width  and  gap  controls. 

• Dielectric  thickness  controls. 

The  test  patterns  for  dielectric  defects,  metal 
continuity  and  polysilicon  and  metal  contacts 
would  contain  large  numbers  of  structures  equiva- 
lent to  the  structures  within  the  LSIC.  The  RTDs 
are  capable  of  evaluating  the  components  of  the 
LSIC  as  each  test  structure  would  be  designed  to 
be  accessible  for  electrial  testing.  They  may 
also  be  packaged  up  for  environmental,  radiation, 
or  accelerated  test  as  components  of  the 
LSIC. 
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t Expand  the  screening  tests  based  on  data 

from: 

• Design  review 

• Fault  analysis 

• Testability  analysis 

• Sensitivity  analysis 

• Characterization 

To  include: 

• Use  of  voltage  stresses 

• Pattern  testing 

• Monitored  dynamic  burn-in  (MIL-M- 
38510  will  be  revised  to  provide 
for  more  than  one  burn-in) 

• Extend  functional  tests  based  on  the 
above  analysis. 

• Re-evaluate  the  visual  inspection 

requirements  and  possible  alternatives. 

These  may  include: 

t Low  power  visual 

• High  power  inspection  of  critical 
and  typical  areas  of  the  chip  on  a 
sample  basis. 

• Use  of  RTDs  from  each  device  wafer. 

• Use  of  accessible  test  devices  on 
each  LSIC  chip. 

Recommendations  and  comments  from  the  participants 
in  this  workshop  will  be  most  welcome. 
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SOME  PRACTICAL  PRINCIPLES  TO  ACHIEVE 
HIGHER  HYBRID  MICROCIRCUIT  RELIABILITY 

Jerome  Fishel 

President,  Circuit  Technology  Incorporated 


processing  a sample  of  each  lot  of  material  received 
through  the  manufacturing  sequence,  we  can  deter- 
mine whether  such  anomalies  will  arise. 

Latent  Problems  with  Incoming  Material 


Introduction 


The  following  suggestions  have  proven  to  be 
important  when  the  objective  is  to  manufacture 
relatively  small  runs  of  hybrid  microcircuits  with 
high  reliability. 

State  of  the  Art  Technology 

Do  not  let  the  Customer  push  you  into  situations 
where  your  design  and/or  manufacturing  process  is 
new.  Stay  well  within  those  design  concepts  and 
good  manufacturing  practices  that  you  are  well 
acquainted  with  and  have  proven  successful. 

Mature  Devices 


Use  only  those  devices  which  have  been  mass  pro- 
duced by  two  or  more  Suppliers  for  a minimum  of 
two  years.  While  your  restraint  may  annoy  some 
designers,  it  will  insure  continued  supply  and 
quality. 

Development  Selection 


While  some  modest  selection  of  some  parameters  is 
acceptable,  avoid  highly  selected  components. 

End  of  Life 


The  circuit  designs  must  be  such  that  performance 
will  not  suffer  even  when  a large  number  of  the 
devices  used  have  reached  an  end  of  life  values. 

100  Percent  Electrical  Inspection  of  Devices 


It  is  important  to  plan  ahead  so  that  all  devices 
to  be  used  can  be  thoroughly  tested  over  the  oper- 
ating temperature  range  prior  to  acceptance.  This 
initial  investment  will  result  in  a major  improve- 
ment in  the  manufacturing  yield  and  the  quality. 

Printed  Versus  Chip  Resistors 

Stay  with  resistors  that  are  screened  and  fired  on 
the  substrate.  Purchased  thin  film  resistor  chips 
have  proven  troublesome. 

Circuit  Partitioning 

It  is  essential  that  each  package  contains  a func- 
tion that  can  be  electrically  characterized  and 
tested  properly.  Try  to  avoid  highly  specialized 
Instrumentation  or  correlation  problems  will  result. 

Substrate  Partitioning 

Avoid  excessive  complexity  by  using  two  or  more 
substrates  in  one  package.  This  avoids  a number  of 
problems  associated  with  large  substrates.  It  also 
provides  an  opportunity  to  pretest  each  substrate 
section  prior  to  committing  it  to  its  package. 

Process  Compatibility  of  Components 

It  has  been  our  experience  that  incoming  material 
that  appeared  quite  normal  proved  to  have  some 
subtle  defects  when  used  in  manufacturing.  By 


One  must  understand  each  device  used  well  enough  to 
anticipate  what  latent  problems  are  typical  and  to 
devise  a specific  accelerated  test. 

Lot  Integrity  of  Incoming  Material 


Wherever  possible,  purchase  semiconductor  devices 
as  uncut  wafers.  Each  wafer  becomes  a homogeneous 
lot  and  can  be  readily  reprobed,  and  characterized. 
Capacitors,  packages,  preforms  and  other  such  mate- 
rial should  also  have  lot  integrity. 

Conductor  and  Dielectric  Printing 

To  insure  good  conductivity  and  adequate  gold  for 
eutectic  bonding,  conductor  and  bonding  pads  should 
be  double  printed.  To  avoid  pin  holes  and  thin 
spots  in  the  dielectric,  double  or  triple  printing 
of  dielectric  is  advisable. 

Resistor  Trimming 


Sand  abrasion  is  preferred  to  insure  freedom  from 
damage  in  the  kerf.  Laser  trimming  is  useful  to 
perform  modest  "active  trims"  where  only  a small 
amount  of  resistor  material  is  to  be  removed. 

Production  Yields 


The  design  and  manufacturing  procedure  should  be 
such  that  the  production  yield  will  be  acceptable 
without  the  need  for  any  significant  repairs.  A 
well  designed  product  using  the  proper  material  and 
well  thought  through  manufacturing  processes  should 
result  in  high  yield.  Iam  not  impressed  with 
people  who  claim  that  their  process  allows  one  to 
make  repairs  easily. 

Process  Forgiveability 

Accept  the  premise  that  we  will  have  high  labor  con- 
tent and  human  beings  are  less  than  perfect.  The 
manufacturing  process  must  have  considerable  margin 
for  error.  One  must  study  each  manufacturing  step 
to  determine  where  the  human  operator  most  often 
may  make  an  error. 

In-Line  Inspection 

It  is  necessary  to  be  as  diligent  in  the  development 
of  100  percent  Inspection  methods  as  one  is  diligent 
in  developing  the  manufacturing  process.  Inspection 
must  be  done  immediately  and  the  result  made  avail- 
able immediately  so  that  effective  corrective  action 
can  be  taken. 

Die  Attach 

Use  gold  silicon  eutectic  die  bonding  wherever  pos- 
sible. Use  gold  filled  epoxy  where  necessary  to 
provide  a mechanical  bond.  Avoid  passing  current 
through  epoxy. 

Capacitor  Attach 

Side  braze  all  capacitors  whose  length  is  100  mils 
or  less.  For  larger  capacitors,  use  non-conductive 


205 

; PRECEDING  PADS  BUMC-NOT  flLMED 


epoxy  for  mechanical  attachment  to  the  substrate 
and  wire  bonding  for  electrical  connection. 

Cleaning  and  Push  Test 


Also,  the  test  should  continue  for  a minimum  of 
three  thousand  hours. 


1 


Wherever  possible,  a small  tool  should  be  used  to 
apply  a modest  pressure  on  all  attached  devices. 
Poorly  attached  devices  can  be  detected  in  this 
fashion.  Also,  a small  tool  can  be  used  to  remove 
any  partially  attached  particles  from  around  each 
device. 

Substrate  to  Package  Attachment 

To  insure  low  thermal  resistance  and  mechanical 
strength,  gold/germanium  or  gold/tin  should  be  used 
to  attach  the  substrate  to  the  package.  Where  the 
substrate  is  to  be  attached  to  vertical  leads,  it 
should  be  fit  over  the  pins  and  the  same  materials 
can  be  used  for  attachment.  It  is  understood  that 
the  substrates  will  have  high  adhesion  material 
printed  on  the  backs  and  around  the  holes. 

Wire  Bonding 


A 100  percent  pull  test  should  be  performed  on  all 
wires  to  a level  of  two  or  three  grams,  depending 
on  the  device  and  wire  size. 

Preseal  Bake  and  Sealing 

All  devices  should  be  thoroughly  baked  for  a mini- 
mum of  six  (6)  hours  at  150°C  in  vacuum.  Sealing 
should  be  accomplished  by  a welding  process  without 
any  preforms  in  a dry  box  which  is  controlled  to 
for  less  than  30  ppm  water  vapor. 

Gas  analysis  should  be  performed  on  a sample  of 
each  sealing  lot. 

Sequential  Burn-in  to  Zero  Failure 

The  burn-in  should  be  a sequence  in  which  the  cir- 
cuits are  tested  after  each  interval.  The  criteria 
should  be  such  that  a given  lot  must  go  through  two 
cycles  in  the  sequence  without  any  failures.  In 
addition,  a maximum  percent  defect  allowable  should 
be  instituted. 

Destructive  Physical  Analysis 

Just  prior  to  shipment,  one  percent  of  the  lot 
should  be  subjected  to  a thorough  DPA  to  insure 
that  parts  have  not  been  damaged  during  sealing, 
environmental  test  and  burn-in. 

PINO  Test 


While  PIND  test  does  detect  particles,  it  is  very 
inconsistent  (non-repeatable)  and  may  be  rejecting 
circuits  which  have  harmless  particles.  I would 
suggest  we  try  to  refine  this  procedure  to  overcome 
these  limitations  or  consider  the  Mann  Test  which 
subjects  the  part  to  vibration  and  shock  while 
operating,  and  detects  anomalies. 

Life  Test 


I believe  the  operating  life  test  should  exactly 
simulate  the  actual  use  conditions  both  electri- 
cally and  mechanically.  This  may  involve  frequent 
interruptions  of  power  or  power  surges,  changes  of 
environmental  temperatures  and  variations  of  load. 


Step  Stress  Life  Test 

A series  of  step  stress  conditions  should  be  applied 
to  a second  group  of  parts  to  induce  failures. 

Analysis  of  these  failures  must  determine  whether 
normal  things  have  happened  or  whether  we  have 
uncovered  a design  or  device  fault  which  needs 
correction. 

In  conclusion,  it  is  my  opinion  that  a hybrid  cir- 
cuit, like  any  other  manufactured  component  can 
provide  a useful  function  in  a reliable  fashion  if 
designed  and  manufactured  properly. 

i 
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JPL  HYBRID  EXPERIENCE 

John  deJong 
Jet  Propulsion  Lab 

• Mariner  Mars  1971  (1968) 

Transistor  arrays 
Diode  arrays 

• Viking  lander  GCMS  instrument  (1969) 

J FET  analog  commutator  switches 
Transistor  arrays 

Diode  arrays 
Operational  amplifiers 
MOS  clock  drivers 

• Mariner  Venus/Mercury  1973  (1970) 

J FET  analog  commutator  switches 
Transistor  arrays 

Diode  arrays 

Plated-wi re-memory  digit/word  drivers 
and  sense  amps 

• Viking  orbiter  (1971) 

Hi  speed  ADC  ladder  switches 
J FET  analog  commutator  switches 
Transistor  arrays 
Diode  arrays 

Plated-wi re-memory  digit/word  drivers 
and  sense  amps 

• Voyager  (1973) 

Transistor  arrays 
Diode  arrays 

Large  scale  digital  hybrid  memories 

• Vortex  instrument  (1976) 

Large  scale  digital  hybrid  system 

• Galileo  (1978) 

Transistor  arrays 
Diode  arrays 

CCD  camera  electronics 

Large  scale  digital  hybrid  memories 
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Voyager  Hybrid  Memory  Chip  Screening 

• Standard  CD4000A  components 

• Dice  procured  from  RCA  with  trace- 
ability  to  the  evaporator  run 

• SEM  requirements  were  GSFC  31 1 -PI 2A 
modified  as  agreed  by  RCA  and  NASA 

• Visual  requirements  were 

MIL-STD-883,  Method  2010,  Level  A 

(included  100  percent  JPL  die  sort 
inspection) 

• Electrical  testing 

• STD  wafer  probe  by  RCA 

• DC  parameters  by  Teledyne  (die 
level  at  25°C) 

• Propagation  delay  by  Teledyne 
(die  level  at  25°C) 

• Post  electrical  visual  die  sort  per 

MIL-STD-883,  Method  2010,  Level  A 

• CD4061  RAM  components 

• Dice  procured  to  commercial 
quality  criteria 

• No  wafer  traceability 

• Dice  visual  requirements 
were  MIL-STD-883, 

Method  2010,  Level  B modi- 
fied (as  defined  by  RCA) 

• JPL  visual  die  sort 
inspection  provided  seg- 
regation of  procured  die 
into  Level  A,  Level  B.and 
Level  C categories 

• Electrical  testing  (visual  A 
components) 

• Dc  parameters  by  Teledyne 
(die  level  at  25°C) 

• Ac  parameters  by  Teledyne 
(die  level  at  25°C) 

• Gapat  pattern  testing 

• Read  access  time 
measurement  testing 

• Post  electrical  visual  die 
sort  per  MIL-STD-883, 

Method  2010,  Level  A 

Galileo  Hybrid  Memory  Chip  Screening  Plan 

• Standard  CD4000B  components 

• Wafer  traceability  to  evaporator 
run  (SEM) 
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• RAD  harness  verification  of  product  directly  assembled  into  the  hybrid,  thus  reducing 

line  potential  component  handling  problems  and  hybrid 

assembly  cost 

• Sampling  program  or 

• Verification  of  process 

• Visual  requirements  to  MIL-STD-883, 

Method  2010,  Level  A (JPL  audit) 

• Electrical  testing 

• Wafer  probe  by  manufacturer 


t 


• Functional  testing  after  hybrid 
assembly  by  hybrid  manufacturer 


I 


1024-bit  status  CMOS  RAM 


• Dice  procured  from  manufacturer 
with  traceability  to  diffusion  runs 

e Wafer  slice  to  verify  wafer  metal i- 
zation  (SEM) 

• 3 packaged  parts  from  each  wafer  to 
verify  functionalism  and  leakage 
current  after  radiation 

• Wafer  probe  utilizing  the  manufac- 
turer's packaged  component,  pre  and 
post  burn-in  tapes 


• Visual  inspection  to  MIL-STD-883, 
Method  2010,  Level  A 

Comparison  of  Voyager  Versus  Galileo  Approach 


Vo.yaqer  F0S 

Galileo  CDS 

Total  fabricated 
bits 

1,048,576 

3,686,400 

Total  dice 
procured 

16,500 

4,000 

Dice  cost 

$288,750 

$101,000  (est) 

Dice  testing  cost 
at  hybrid 
contractor 

$21 ,000 

Included  above 

Dice  testing 
program 

Minimal  - 
(dc  test 
plus  Galpat 
and  read 
access  time 
measurements 
at  25°C) 

Good  - (full 
parametric  ac 
and  dc  testing 
at  25°C  - com- 
plete wafer 
traceability 
Including  radia- 
tion harness 
verification 
and  SEM  analysis) 

Conclusion/Recommendation 

• Semiconductor  chip  manufacturer's  are 
able  to  provide  improved  screening 
options  at  a lower  cost  than  screening 
and  testing  options  which  utilize  a 
hybrid  manufacturer's  dice  testing 
capabilities 
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• The  dice  procured  from  the  manufacturer 
should  be  of  a quality  that  can  be 


CLASS  "S"  HYBRID  MICROELECTRONICS  COMMITTEE 

Donald  L.  Fresh 
Technical  Staff 
The  Aerospace  Corporation 


SURVEY  CONDUCTED  BY 


Col.  W.L.  Schlosser,  Director 
Acquisition  Support 


Contamination  or 
Particles 
(1&3HI 


Dr.  M.T.  Weiss,  General  Manager 
Electronics  & Optics  Division 
The  Aerospace  Corporation 


Interconnact 

125.3%) 


RESULTS  PRESENTED  TO 


Lt.  Gen.  T.W.  Morgan 
Commander 
SAMSO 


FAILURE  MODES 


To  determine  the  status  of  hybrid 
technology  in  industry  and  the  implica- 
tions of  this  technology  on  SAMSO 
programs 

To  investigate  the  advantages  and  dis- 
advantages of  hybrid  usaqe  with  particu- 
lar emphasis  on  reliability 

To  make  recommendations  for  future 
actions  to  assure  hybrid  reliability  for 
space  applications 


Technical  and  Reliability  Considerations 


IC  and  transistor/diode  chips 


Substrates 


Chip  and  wire  attachment 


Packages 


Fabrication  cleanliness 


Particle  contamination  control 


Usage  of  Hybrids 


Testing 


• Rapidly  increasing  use  of  commercial  and 
consumer  electronics 


Summary  Conclusions  and  Recommendations 


• Increasing  use  in  space  systems 


e Increasing  use  of  hybrids  in  SAMSO  sys 
terns  inevitable 


• DSCS  III  hybrid  usage 


• Size  and  weiqht  are  primary  driving 
factors 


• Per  spacecraft 


Desion  flexibility,  technical 
requirements,  cost  and  reliability 
are  important  factors 


• GE  fabrication 


CTI  fabrication 


Hybrid  usage  should  result  In 
Increased  performance  for  SAMSO 
systems 


• Teledyne 
fabrication 


SAMSO  must  take  active  steps  to  assure 
compatibility  of  hybrid  usage  with  space 
reliability  and  performance  requirements 

• SPO  should  review  all  aspects  of 

hybrid  development,  vendor  selection 
fabrication  and  test 


• Space  Shuttle  Orbiter 

• Used  in  37  subsystems 


• 35  vendors 


0V101  total  - 12,580  hybrid 
units 


• Testing  of  complex  hybrid  modules 
may  require  special  development 

• SPO  should  control  hybrid  complexity 
to  be  compatible  with  development 


0V102  total  - 15,982  hybrid 
uni  ts 


2.  Qualification/Screening  Procedures 


schedule,  supplier  experience,  cost 
constraints,  rework  criteria, 


testabil ity 

J.  Hilibrand** 

RCA/GS 

P.  Dick* 

GE/VF 

t SAMSO/ NASA/ RADC  should  develop 

D.  Cole 

GE/U 

improved  specifications  and  stan- 

J. Farrell 

RADC 

dards  for  space  quality  Class  S 

D.  Hill 

DESC 

hybrids 

R.  Rohal 

NASA/L 

• Government/industry  ad  hoc 

3.  Test  Method-Active  Chips 

committee  on  hybrid  micro- 

electronics underway 

P.  Eisenberg** 

Northrop 

J.  deJong* 

JPL 

Committee  on  Hybrid  Microelectronics 

P.  Dick 

GE/VF 

J.  Hilibrand 

RCA/GS 

Define  the  technical  requirements  and 
develop  the  documentation  for  Class  S 

D.  Loescher 

Sandia 

hybrid  microcircuits  including: 

4.  Test  Method-Passive  Chips 

• General  specification 

E.  Zimmerman** 

Boeing 

D.  Cole* 

GE/U 

• Detailed  specification  format 

E.  Thompson* 

LMSC 

W.  Berger 

Ford 

Layout  and  design  requirements 
Manufacturer  certification 


5.  Test  Method- Internal  Visual  (and  Others) 


D.  Laotiz** 

TRW 

• Qualification  and  quality  con- 

E.  Cormier* 

GD/C 

formance  requirements 

C.  Murphy 

RI 

• Screening  procedures 

6. 

MFG  Certification 

• Unique  test  methods 

C.  Murphy** 

RI 

S.  Caruso* 

NASA/M 

21  members;  Chairman,  D.L.  Fresh 

M.  Gaudiano 

NASA/J 

The  Aerospace  Corporation 

1 

7. 

Packages  (and  Substrates) 

AFML 

1 

DESC 

1 

L.  Schneider** 

TRW/MM 

JPL 

1 

P.  Eisenberg* 

Northrop 

NASA 

3 

E.  Cormier 

GD/C 

RADC 

1 

SAMSO  Contractors 

12 

Schedule  (Hybrid  Microelectronics 

Committee) 

Subcommi ttees 

1.  General  Specification 


April  4 and  5 
April  5 


J.  Farrell** 

L.  Schneider* 
S.  Caruso 
D.  Hill 

J.  Whittington 


RADC 

TRW/MM 

NASA/M 

DESC 

HAC 


April  7 - June  18 
June  19,  20,  and  21 


First  meeting  of  committee 
Initial  meetings  of 
subcommittees 
Subcommittee  activity 
Second  meeting  of  committee 


1A.  Layout/Design/Construction 


E.  Cormier** 
D.  Loescher* 
0.  Lapitz 


GD/C 

Sandia 

TRW 


IB.  Organics/Polymers 


S.  Caruso** 
R.  Rohal* 
AFML 


••Chairperson 

•Vice-Chairperson 


NASA/M 

NASA/L 

AFML 
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IMPLEMENTATION  OF  THE  COMMANDER'S  POLICY 
SUPPLIER  SURVEILLANCE 

Warren  Geller 
Staff  Engineer 

Lockheed  Missiles  & Space  Co.  Inc. 

• Electronic  part  complexity  increased 

• Part  specification  improved 

• New  SAMSO  policy  to  manage  electronic 

parts  procurement 

• LMSC  support  to  SAMSO 
Electronic  Part  Complexity  has  Increased 

Part  Type  Complexity 

Resistor  Basic  part 

Transistor  2 diodes 


• Manage  discrepant  hardware 

t Identify  and  correct  problems  at  the  earliest 
point 

Monitored  Line  Service 

LMSC  arrangement  with  AF  to  provide  surveillance 
(monitoring)  for  Government  contractors 

Plan  and  parts  list  available 

LMSC  History  in  Surveillance 

Monitored  line  parts  program  - 5 years 

Prepared  200  drawings  and  specifications  for  tran- 
sistors and  IC's 

Delivered  170,000  parts  to: 

15  major  LMSC  programs 
20  subcontractors 


Microcircuits: 


SSI  Up  to  12  gates  (48  transistors) 

MSI  12-100  gates  (400  transistors) 

LSI  100-2000  gates  (8000  transistors) 

Specification  Complexity  has  Increased 
MIL-S-19500  Transistors  and  Diodes 


JAN 

JAN  TX 
JAN  TXV 
Class  S 


MIL-STD-750 


MIL-M-38510  Microcircuits 


Class  B & C 
Class  A 
Class  S 


MIL-STD-883 


Hybri ds 

Specifications  under  development 


Program  continuing  and  evolving  to  Class  S 
Observed  effectiveness  - cost/performance 


SAMSO  COMMANDER'S  POLICY  - SUPPLIER  SURVEILLANCE 
BOX  LEVEL  TEST  FAILURES  (ONE  PROOltAM) 

• PARTS  PURCHASED  IN  SAME  TIME  FRAME 

• PARTS  TESTED  IN  SAME  EQUIPMENT 


% 

FAILURES 


RELIABILITY 
GAIN  FACTOR 


(A8SUMING  1 MLP 
FAILURE) 


DIGITAL  IC’S 
MONITORED 
NON- MONITORED 


LINEAR  IC'S 


TRANSISTORS 


5,  270 
1,877 


17.459 

2,064 


New  SAMSO  Policy  - Supplier  Surveillance 

Technique  to  improve  management  of  electronic 
parts  for  space  use 

Recognizes  need  for  field  personnel  i-.volvement 
MIL-STD- 1 546  Appendix  B (proposed) 

Supplier  Surveillance  Functions 

• Provide  real-time  support  to  the  supplier 

• Interpret  specification  (subjective 

requirements) 

• Direct  failure  analysis  and  corrective  action 

• Control  part  configuration 

• Recommend  specification  improvements 


Summary 

Electronic  part  complexity  approaching  equipment 
level 

Part  specifications  undergoing  continual  upgrading 

Surveillance  policy  developed  to  better  manage 
modern  part  procurement 

LMSC  history  verifies  effectiveness  of  surveillance 

Monitored  line  service  available  to  contractors 

The  subject  of  today's  workshop,  Microelectronics/ 
Hybrids,  represents  the  state-of-the-art  in  elec- 
tronic devices.  As  the  complexity  of  SAMSO  mis- 
sions increases,  the  use  of  these  devices  will 
become  a practical  necessity.  A major  problem 
facing  us  is  converting  the  design  requirements 
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for  these  new  devices  into  reliable,  space-worthy 
hardware. 

The  Commander  of  SAMSO  has  recognized  that  what 
today  we  are  calling  a "part,"  yesterday  we  called 
an  electronic  box.  He  suggests  that  we  must  begin 
to  extend  some  of  the  management  techniques  pre- 
viously reserved  for  electronic  equipments  to  the 
manufacture  and  testing  of  these  complex  parts. 

The  technique  developed  for  the  advanced  manage- 
ment of  electronic  parts,  called  "line  monitoring" 
or  "supplier  surveillance"  has  been  described  in  a 
new  SAMSO  policy.  The  requirements  for  supplier 
surveillance,  to  be  imposed  contractually  at  a 
future  date,  are  defined  in  a recently  released 
draft  of  Appendix  B to  MIL-STD- 1546,  titled,  "Sup- 
plier Surveillance."  For  the  past  5 years  LMSC 
has  been  managing  the  Monitored  Line  Parts  Program, 
performing  the  supplier  surveillance  function 
des'iibed  in  Appendix  B.  Having  delivered  over 
170,000  transistors  and  integrated  circuits  to 
15  LMSC  major  programs  and  over  20  LMSC  subcon- 
tractors, it  was  considered  appropriate  that  we 
share  our  experience  with  SAMSO  and  its  contractors. 

The  need  for  surveillance  derives  from' the  com- 
plexity and  depth  of  requirements  found  in  recent 
semiconductor  specifications.  We  have  exceeded 
the  point  where  thicker  specifications  can  affect 
the  reliability  of  our  hardware  unless  we  supple- 
ment the  specifications  with  personal  attention. 

In  addition  to  providing  real-time  support  to  the 
supplier  in  specification  interpretation,  on-site 
personnel  are  needed  to  direct  failure  analysis  and 
corrective  action,  to  manage  hardware  not  complying 
with  the  letter  of  the  specification  but  useful  to 
the  user,  and  to  provide  a direct  communication 
line  between  the  users  and  the  manufacturing  and 
testing  personnel.  The  objective  is  to  manage 
parts  with  paper  and  people. 

LMSC  experience  has  revealed  that  the  monitoring 
team  must  contain  3 segments:  1)  resident  monitor- 
ing personnel;  2)  centralized  technical  and 
administrative  support;  and  3)  supplier  partici- 
pants. The  resident  personnel  includes  a quality 
engineer  supported  by  one  or  more  inspectors 
and  a parts  engineer  to  provide  technical  direc- 
tion. This  group  is  fully  capable  of  managing  the 
routine  tasks  of  inspecting,  testing,  and  shippinq 
products.  The  central  group  provides  in-depth 
technical  support  when  problems  occur,  prepares 
specifications  and  drawings  and  completes  the  com- 
munication line  to  the  users.  It  must  be  recog- 
nized early  that  the  system  cannot  function  unless 
the  monitoring  team  is  adequately  supported  by  sup- 
plier personnel.  They  must  willingly  be  members  of 
the  team. 

User  specifications  and  drawings  are  normally  not 
used  in  the  supplier's  assembly  and  test  areas  in 
preference  to  their  own  documentation.  Again, 
recognizing  the  complexity  of  modern  part  require- 
ments, it  becomes  critical  that  a careful  review 
be  made  of  supplier  procedures  and  specifications 
to  verify  that  hardware  is  being  manufactured  and 
tested  in  accordance  with  user  requirements. 

The  LMSC  Monitored  Line  Program  has  proven  to  be 
extremely  successful.  The  failure  rate  in  equip- 
ment has  been  approximately  0.1  percent  and  the 
causes  of  two-thirds  of  these  failures  have  since 


been  the  subject  of  corrective  action.  Only  one 
failure  h'.s  occurred  at  a testing  level  higher  than 
box  final  test.  There  have  been  no  failures  at 
system  level  testing  or  in  end  use.  Cost  and 
schedules  have  been  improved  by  the  low  rate  of 
failures. 

Recently,  the  SAMSO  Launch  Vehicle  Programs  auth- 
orized use  of  LMSC  Monitored  Line  parts  in  their 
Systems.  A detailed  plan  has  been  prepared  for 
making  a Monitored  Line  Service  available  to  gov- 
ernment agencies  and  their  contractors.  A con- 
tract is  currently  under  negotiation  to  implement 
this  plan  for  the  use  of  SAMSO. 
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• More  effective  coordination  and  interface 
with  parts  supplier(s) 

• Reduce  the  proliferation  of  program  com- 
ponent specifications 

• Establish  the  most  cost  effective 
approach  for  total  program  requirements 

• Better  application  of  current  procure- 
ment techniques 

Anticipated  Advantages 

• Reduce  number  of  specifications  required 

• More  effective  use  of  personnel,  i.e., 
engineering,  quality  assurance, 
purchasing 

• Lot  test  charge  savings 

t Improvement  in  total  program  scheduling 

through  program  allocation  of  parts 

• Maximize  importance  of  program  require- 
ments to  participating  manufacturers 

• Establish  the  most  economical  unit  prices 
Areas  of  Concern 

• How  are  requirements  identified  and 
consolidated? 

• How  are  parts  shortages  handled  when 
manufacturer  cannot  meet  total  program 
needs? 

• Spec  preparation,  negotiation,  changes 

• How  are  parts  ordered  by  the  participat- 
ing buyers? 

• Prime  contractor  liability 

• Pricing  of  requirements 
Parts  Management/Parts  Control  Board 

• Identification  of  part  candidates 

• Selection  of  device  prime 

• Optional  participation 
Responsibilities  of  Device  Prime 

• Specification  coordination 

• Configuration  baseline 


• Source  inspection 

• Radiographs 

• SEM  photos 

• Lot  test  data 

a Discrepant  parts  disposition 

• Coordinated  procurement  proposal 

• Parts  price 

• Data  price 

• Purchase  contract  placement  window 

• Shipment  dates 

• Price  negotiation 

• Total  quantity 

• Lot  charges 

Participating  Buyer  Responsibilities 

• Purchase  contract 

• Requirement  changes 

• Specification  change  coordination 

t Review  test  data 

• Accept/reject  parts 
Liability  Disclaimer 

It  is  agreed  the  device  prime  will  have  no  liabil- 
ity for  part  deficiencies,  part  delivery  slides  or 
other  concerns  related  to  coordinated  procurement. 
The  subcontractor's  sole  recourse,  if  needed,  shall 
be  through  its  contractual  relationship  with  the 
parts  supplier. 

Concl usions 

• Objectives  will  be  met 

a More  effective  coordination  with 

suppliers 

• Reduce  specification  proliferation 

• Establish  most  cost  effective 
approach 

• Improve  procurement  technique 
Recommendations 

• Adopt  coordinated  procurement  for  custom 
specification  parts 

• Preserve  participating  subcontractor 
responsibility 


Schedule 
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LSI/HYBRID  WORKSHOP 

PROPOSED  CONCLUSIONS  AND  RECOMMENDATIONS 


! 


Note:  Workshop  attendees  are  asked  to  indicate 
either  agreement  or  disagreement  with  each 
of  the  conclusions  and  recommendations 
listed  by  writing  the  word  AGREE  or  DISAGREE 
in  the  adjacent  left-hand  margin.  Addi- 
tional comments,  conclusions  or  recommenda- 
tions are  encouraged  and  should  be  written 
in  at  the  end  of  this  listing. 

CMOS/SOS  RELIABILITY  STATUS 

Conclusions : 

1.  The  CMOS/SOS  technology  has  made  signifi- 
cant advances  during  the  past  two  years, 
and  it  is  in  production  status  at  RCA  and 
can  be  qualified  for  applications  in 
space. 

2.  CMOS/SOS  is  the  leading  technology  for 
VLSI  functions  as  a result  of  its  out- 
standing speed  power  product  and  high 
packing  density. 

3.  CMOS/SOS  devices  have  demonstrated  long- 
term stability  at  25  DGR.  C. 

4.  Mask  compatible  second  sourcing  is 
available. 

5.  RCA  is  fully  committed  to  supplying 
space  qualified  CMOS/SOS  parts. 

Recommendations : 

The  aerospace  industry  should  be  making  more  use 

of  CMOS/SOS  technology. 

i2l/lsi  in  military  applications 

Conclusions: 

2 

1.  I L is  a viable  LSI  technology  for 
military  and  space  applications. 

2 

2.  I L can  meet  temperature  and  radiation 
hardness  requirements  for  space 
applications. 

2 

3.  I L gate  arrays  will  permit  low  volume, 
fast  turnaround  LSI  implementation. 

4.  I2L  failure  rates  of  less  than  10  FITS 
under  normal  stress  are  predicted. 

2 

5.  Advanced  T L has  growth  potential  for 
VLSI  (?50  MHz,  ^104  gates  per  chip). 

Recommendations: 

1.  Provide  incentives  for  LSI/VLSI  utiliza- 
tion in  space  systems. 

2.  Encourage  set-up  of  systems  oriented 
custom  LSI  centers  with  qualified  low 
volume  fabrication  facilities. 

3.  Encourage  development  of  LSI  technologies 
with  high  degree  of  fabrication  com- 
monality to  assure  second  sourcing. 


4.  Establish  procurement  policies  to 
guarantee  life-cycle  support  for  LSI 
parts. 

5.  Develop  user-oriented  LSI  performance 
and  reliability  test  procedures  and 
specifications. 


RADIATION  EFFECTS  ON  LSI 

Conclusions : 

The  two  critical  LSI  problems  are: 

1.  The  evolution  of  high  performance  arrays 
at  the  expense  of  internal  design  mar- 
gins on  critical  parameters  which  leads 
to  increased  radiation  vulnerability.  If 
this  trend  continues,  all  hardened  LSI 
will  have  to  be  custom  structures 
significantly  different  than  main-line 
industrial  technology. 

2.  Experimental  evaluation  of  the  worst-case 
radiation  failure  level.  Displacement 
effects,  ionization  effects  and  electri- 
cal overstress  must  be  determined  by  a 
set  of  bias  conditions  and  post- 
irradiation test  vectors  that  can  cost- 
effectively  reflect  the  unique  failure 
mechanisms  and  worst-case  failure  level. 
This  is,  however,  almost  straight- 
forward compared  to  detecting  the  worst- 
case  logic  upset  level  from  a pulsed 
ionizing  radiation  exposure. 

Recommendations : 

1.  SAMSO  must  get  involved  with  the  develop- 
ment of  "essentially  industrial"  LSI 
technology  with  radiation  hardness  as 

a substantial  but  not  dominating  concern. 
Technology  support  should  be  implemented 
in  all  high-performance/high  density  LSI 
technologies  (including  n-MOS,  CCD, 
CMOS/SOS,  digital  I^L,  and  combined 
bipolar  analog/digital  I^L).  The  level 
of  support  must  be  of  sufficient  magni- 
tude and  focus  to  get  and  retain  the 
attention  of  the  US  semiconductor 
industry. 

2.  The  LSI  test  problem  should  be  formulated 
as  a program/goal-oriented  set  of  tasks 
that  can  be  examined  in  detail.  These 
might  include: 

a.  A methodology  to  define  a set  of 
test  vectors  to  establish  worst- 
case  bias  conditions  on  an  MOS/LSI 
array  (CMOS,  CMOS/SOS  or  n-MOS) 
during  ionizing  radiation  exposure; 

b.  txter  ‘ions  of  LSI  test  methodology 
to  iczntify  the  location  of  a failed 
cel • rnd  the  nature  of  the 
radiation-induced  failure; 

c.  Development  of  an  optimum  set  of 
test  vectors  to  detect  transient 
logic  upset  resulting  from  pulsed 
ionizing  radiation  exposure. 
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3.  Test  methodologies  should  be  developed 
for  application  by  system  designers 
rather  than  the  semiconductor  LSI  designs. 
Maximum  use  must  be  made  of  post- 
fabrication performance  parameters  with 
minimum  dependence  on  design  or  fabrica- 
tion parameters. 


HI  REL  APPLICATIONS  OF  CUSTOM  LSI 
Conclusions: 

1.  Custom  LSI  is  applicable  to  high  relia- 
bility systems. 

2.  The  design  development  should  be  done  by 
the  user  based  on  available  IC  technology. 

Recommendations: 

1.  Base  designs  on  well  characterized  stan- 
dard cells. 

2.  Make  extensive  use  of  computer  aids  to 
keep  the  design  cost  down  and  maximize 
confidence  in  the  design. 


DESIGN  TOOLS  FOR  CUSTOM  LSI 

Conclusions: 

The  use  of  custom  CMOS  LSI  parts  (along  with 
microprocessors  and  memories)  is  the  most  reliable 
and  cost-effective  way  to  implement  space  systems. 

Recommendations : 

1.  Special  attention  is  needed  to  provide 
for  custom  LSI  part  design  early  in  the 
prototype  and  testing  phase  of  programs 
so  that  the  life  cycle  cost  benefits  from 
the  use  of  LSI  parts  can  be  achieved 
during  the  production  phase  of  the 
program. 

2.  Custom  LSI  parts  should  be  qualified  on 
a family  basis  through  testing  of  stan- 
dard cell  libraries  and  universal  array 
vehicles  rather  than  by  fully  qualifying 
each  custom  part  individually. 


TESTING  MICROPROCESSORS 

Conclusions : 

1.  Test  patterns  derived  through  both 
logic  and  functional  simulation  offers 
the  highest  confidence  test  for  space 
and  high-rel  applications. 

2.  The  use  of  both  techniques  is  not 
practical  for  all  applications  due  to 
cost  considerations  and  limited  access 

to  the  correct  schematic  (that  is,  unless 
the  pattern  derived  through  logic  simula- 
tion is  provided  by  the  device 
manufacturer). 

3.  The  best  single  method  for  space  applica- 
tion is  functional  simulation. 


MICROPROCESSOR  SLASH  SHEET  DEVELOPMENT 
Conclusions : 

1.  Complete  characterization  testing  must  be 
accomplished  before  a specification  can 
be  written.  Even  then,  significant  prob- 
lems exist  in  defining  certain  specifica- 
tion details  (e.g.,  dynamic  burn-in  cir- 
cuit, accelerated  life  test  conditions, 

p tests,  capacitance  tests,  etc.). 

2.  The  device  has  several  serious  limita- 
tions (e.g.,  lack  of  test  ability,  timing 
problems,  problems  in  running  functional 
tests  rated  frequency,  etc.). 

3.  The  ability  to  perform  functional  and 
switching  tests  is  seriously  limited  on 
several  test  systems  and  even  on  dif- 
ferent configurations  of  the  same  test 
system. 

4.  No  LSI  can  be  expected  to  meet  all  the 
requirements  of  MIL-M- 38510,  Class  S,  as 
they  now  exist. 

Recommendations : 

Because  of  all  the  subtle,  and  not  so  subtle, 
problems  encountered  in  characterizing  and  specify- 
ing microprocessors,  there  is  need  for  a single 
clearing  house  to  collect,  analyze,  and  disseminate 
LSI  information. 


CLmSS  S LSI  PLANS 

Conclusions : 

Existing  Class  S microcircuit  requirements  are 
inadequate  for  LSI. 

Recommendations : 

Expand  Class  S requirements  for  LSI  to  include: 

1.  Design  review 

2.  Built-in  testability 

3.  Certification  of  design  rules,  cell 
libraries  and  CAD 

4.  The  use  of  reliability  test  devices  in 
wafer  lot  acceptance 


HI  REL  HYBRID  MANUFACTURING  CONSIDERATIONS 
Conclusions : 

There  is  a spittle  contamination  problem  which 
effects  hybrids  or  any  large  area  device. 

Recommendations : 

Operators,  inspectors  and  others  coming  in  close 
proximity  of  open  hybrids  should  wear  face  masks. 
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CLASS  S HYBRID  CIRCUIT  COMMITTEE 


LSI  TECHNOLOGY 


Conclusions  and  Recommendations : 

The  increased  use  of  hybrid  microelectronics  in 
SAMSO  systems  is  inevitable,  SAMSO  must  take  active 
steps  to  assure  compatibility  of  hybrid  usage  with 
space  reliability  and  performance  requirements, 
and  SAMSO/NASA/RADC  should  develop  improved  speci- 
fications and  standards  for  space  quality  hybrids. 


IMPLEMENTING  COMMANDER'S  POLICY  FOR 
SUPPLIER  SURVEILLANCE 

Conclusions: 

The  management  techniques  formerly  reserved  for 
equipment  procurements  must  be  applied  to  complex 
electronic  parts  such  as  LSI. 

Recommendations : 

The  SAMSO  Commander's  Policy  as  implemented  in 
proposed  Appendix  B of  MIL-STD-1 546  should  be  given 
immediate  attention. 


COORDINATED  PROCUREMENT  FOR  ISU 
Conclusions: 

Objectives  will  be  met: 

1.  More  effective  coordination  with 
suppliers 

2.  Reduce  specification  proliferation 

3.  Establish  most  cost  effective  approach 

4.  Improve  procurement  technique 
Recommendations : 

1.  Adopt  coordinated  procurement  for  custom 
specification  parts. 

2.  Preserve  participating  subcontractor 
responsibility 


WORKSHOP  SUMMARY 

• Emphasis  on  LSI  and  hybrids 

• Fifteen  presentations  with  discussion 

• Tabulation  of  conclusions/ 
recommendations  commented  on  by 
participants 

• Final  wrap  up  discussion 

• Major  problems  identified  in 

• LSI  technology 

• Custom  LSI 

• LSI  test  and  specification 


k. 


Problem: 

• No  single  LSI  technology  has  demonstrated 
ability  to  meet  all  SAMSO  requirements 

• Apparent  growing  divergence  between 
standard  LSI  circuit  capability  and 
SAMSO  requirements 

Solution: 

• SAMSO  must  continue  to  support  and 
evaluate  multiple  LSI  technologies 

• SAMSO  must  influence  the  design  of  cer- 
tain standard  LSI  circuits 

CUSTOM  LSI 

Problem: 

• Custom  LSI  circuits  offer  significant 
advantages  however  limited  production 
precludes  adequate  characterization 

• Custom  LSI  circuit  design  and  develop- 
ment cost  and  schedules  could  be 
prohibitive 

Solution: 

• Base  custom  designs  on  well  characterized 
computerized  design  systems  and  asso- 
ciated technologies 

• SAMSO  should  establish  approved  custom 
LSI  design  systems/contractors 

LSI  TEST  AND  SPECIFICATION 

Problem: 

• Standard  LSI  circuits,  particularly 
microprocessors  are  extremely  difficult 
to  test 

• Existinq  Class  S microcircuit  require- 
ments are  not  adequate  for  LSI 

Solution : 

• Support  and  expand  evaluation  and  char- 
acterization of  LSI 

• Establish  a single  clearing  house  to  col- 
lect, analyze  and  disseminate  LSI 
information 

t Expand  Class  S requirements  to  include 
LSI 

OTHER  OBSERVATIONS 

• SAMSO  should  promote  the  effective  use 
of  LSI 

• Class  S LSI  requirements  urgently  needed 

e Future  space  systems  will  employ  more 
complex  hybrids 
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Line  monitoring  concept  applicable  to 
hybrids  and  LSI 


• LSI  and  complex  hybrid  circuit  specifica- 
tion and  procurement  more  similar  to 
subsystem  than  part 

i May  be  counter  productive  to  dictate 
technology 
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CRITICAL  ITEM  CONTROL 
Lee  B.  Gray 

Division  Director  Assurance  Management 
Rockwell  International 


The  government  and  the  contractor  are  both  always 
concerned  that  they  may  be  working  "very  accurately 
on  the  wrong  problem."  In  these  days  of  severely 
limited  budgets,  we  must  quickly  zero  in  on  the 
real  potential  problems  and  assure  that  we  are  con- 
trolling that  hardware  that  can  cause  loss  of  life 
or  mission  or  both. 

In  recent  meetings  between  top  Air  Force  officials 
and  company  executives,  the  quality  assurance  func- 
tion has  justly  been  criticized  for  lack  of  innova- 
tion. We  make  damn  sure  it  is  built  like  the  print, 
even  if  the  print  is  wrong.  Quality  audits  conduc- 
ted  by  the  government  and  industry  often  result  in 
satisfactory  reports  only  to  have  a major  failure 
occur  a short  time  thereafter.  Their  usual  response 
is,  "It's  a design  problem."  If  this  is  true,  then 
their  quality  audit  should  have  found  this  "design" 
problem. 

In  order  to  control  "critical  items,"  we  first  must 
define  the  term.  In  the  Shuttle  program,  an  item  is 
critical  if  its  failure  could  lead  to  loss  of  life, 
or  mission,  or  its  redundant  items  cannot  be  checked 
out  prior  to  flight,  or  the  loss  of  the  redundant 
item  is  not  detectable  in  flight,  or  lastly,  all 
redundancy  can  be  lost  by  a single  credible  event. 

On  the  Shuttle  Orbiter  program,  two  percent  of  the 
items  we  build  at  Space  Division  and  49  percent  of 
the  items  we  buy  fall  into  at  least  one  of  the 
above  categories. 

The  Failure  Mode  and  Effect  Analysis  (FMEA)  identi- 
fies the  items  which  are  sumnarized  in  the  Critical 
Items  List  (CIL).  Let's  walk  through  an  actual 
case  of  how  an  assessment  was  made  of  one  small 
critical  item  in  a review  recently  held  to  assess 
the  readiness  of  our  Orbiter  for  its  maiden  space 
flight. 

The  Orbiter's  payload  bay  doors  must  be  opened  on 
orbit  each  flight  in  order  to  perform  its  mission. 
These  doors  cover  the  60'  by  15'  payload  bay 
(Figure  1).  These  doors  are  operated  by  a complex 
mechanical  drive  and  latch  mechanism.  Prior  to 
entry,  at  the  conclusion  of  the  mission,  these 
doors  must  close.  Failure  to  do  so  would  cause 
mission  and  crew  loss.  One  item  which  could  cause 
this  condition  is  the  torque  shaft  power  drive. 

The  CIL  states  that  the  cause  of  this  failure  could 
be  stress  corrosion,  fatigue  or  overstress  due  to 
binding.  All  of  these  potential  causes  must  be 
investigated  but  we  will  only  follow  the  last  cause, 
"overstress  due  to  binding."  (Figure  2.) 

The  Quality  Engineer  (QE)  must  then  gather  all  of 
the  specifications  and  drawings  concerning  this 
power  drive.  He  develops  a drawing  and  specifica- 
tion tree  until  he  gets  down  to  the  component  whose 
failure  would  be  the  cause  identified  In  the  CIL. 

The  QE  must  then  prepare  a questionnaire  regarding 
this  critical  item  and  its  unique  failure  mode. 

Note  that  there  is  no  standard  questionnaire  or 
audit  check  list.  Each  must  be  customized  if  you 


are  serious  about  getting  to  the  real  problem. 

(Figure  3.)  The  QE  then  has  a questionnaire  review 
session  with  the  responsible  design  and  reliability 
engineers.  After  incorporating  their  comments,  he 
is  ready  to  hit  the  road  to  do  a product 
assessment.  This  in-house  preparation  time  has 
taken  from  six  to  ten  days  depending  on  product 
complexity. 

At  the  subcontractor  he  will  average  three  to  five 
days  of  product  assessment  (one  half  the  time  spent 
in-house).  He  spends  his  time  on  the  floor  looking 
at  the  hardware  and  the  shop  paper  that  is  produc- 
ing it,  not  Q.A.  plans  and  procedures.  He  is  asking, 
"What  did  you  do?",  not  "What  were  you  supposed  to 
do?"  Our  QE  discovers  that  the  sleeve  bearing  is 
only  being  lubricated  on  the  outer  diameter  but  not 
on  the  inner  diameter.  This  is  according  to  print! 

In  that  the  failure  of  this  and  similar  bearings  in 
the  payload  bay  door  system  could  cause  loss  of 
crew  and  mission,  it  is  obvious  both  inner  and  outer 
diameter  should  be  lubricated.  This  means  change 
the  print!  A normal  quality  audit  would  have  found 
the  process  OK  in  that  it  met  the  print. 

When  the  QE  returns  from  his  assessment,  he  briefs 
the  directors  of  quality  and  reliability  and  makes 
any  recommendations  for  corrective  action.  Those 
actions  deemed  necessary  to  verify  that  the  failure 
mode  cannot  be  built  into  the  hardware  are  then 
implemented  with  the  supplier. 

This  product  assessment  process  responds  to  the 
criticism  of  top  military  and  industry  executives. 

The  assessor  (auditor)  must  be  an  "expert"  in  the 
product  to  be  assessed.  I have  been  audited  by 
auditors  that  wouldn't  recognize  my  product  if  they 
fell  over  it  in  the  hall!  The  product  assessment 
method  becomes  a "rifle  shot"  technique  rather  than 
the  "shotgun"  technique  which  results  from  the  stan- 
dard audit  questionnaire.  It  means  an  increased 
preparation  time,  but  less  time  at  the  supplier. 

It  is  a cost  effective  method  in  that  you  are  no 
longer  working  "the  wrong  problem  very  accurately." 
You  are  now  solving  the  "real  problem"  before  it 
occurs  in  flight. 


FAILURE  CAUSE  ASSIGNMENT 
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QUESTIONAIRE  DEVELOPMENT 


I SLEfl816°MR)ING  |*FAILURE  CAUSE:  BINDING 

SAMPLE  QUESTIONS  & ANSWERS 

Q-  HOW  IS  DUAL  ROTATION  OF  BEARING  ASSURED? 

A-  BY  MACHINING  BEARING  FOR  WORST  CASE  TOLERANCE  OF  0.0005 

O-  DOES  INSTALLATION  PROCEDURE  SPECIFY  LUBE  ON  OUTSIDE 
DIAMETER  OF  BEARING  AND  INSIDE  DIAMETER  OF  HOUSING? 

A-  NO.  ONLY  INSIOE  DIAMETER  OF  HOUSING  IS  LUBRICATED * 

Q-  HOW  IS  BEARING  CLEANED  TO  PREVENT  CONTAMINATION? 

A-  PARTS  ARE  CLEANED  TO  300  LEVEL  & ASSEMBLED  IN  100.000 
CLEAN  ROOM 

NOTE:  ALL  OF  THE  ABOVE  OPERATIONS  ARE  VERIFIED  BY  QUALITY  CONTROL 

* AS  A RESULT  OF  THE  CONCERN  FOR  INSUFFICIENT  LUBRICATION. 
PROCEDURE  WAS  CHANGED  TO  ADD  REQUIREMENT  FOR  LUBRICATION 
OF  OUTSIDE  DIAMETER  OF  BEARING 
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TAILORING  THE  QA  ROLE  FOR  RDT&E 

Richard  H.  Flohr 
Manager,  SSD  Product  Assurance 
Lockheed  MSC 

My  presentation  today  begins  with  the  Quality  role 
as  applied  in  the  LMSC  Space  Systems  Division  (SSD) 
for  development  programs.  I have,  for  the  purpose 
of  this  presentation,  assumed  that  MIL-Q-9858A 
would  be  the  basic  contractual  specification 
imposed  on  our  "development"  product. 

MIL-Q-9858A  requires: 

• The  establishment  of  a quality  program  by 
the  contractor  to  assure  compliance  with 
the  requirements  of  the  contract. 

• The  Quality  program  shall  be  subject  to 
review  by  the  Government  Representative 
and  is  subject  to  the  disapproval  of  the 
Government  Representative  whenever  the 
contractors'  procedures  do  not  accomplish 
their  objectives. 

We  must,  therefore,  be  responsive  to  MIL-Q-  and  any 
other  contractual  quality  specification  in  response 
to  RFPs  because  the  danger  of  nonresponsiveness 
and  possible  disqualification  in  the  source  evalu- 
ation process.  Let's  face  it,  this  is  a strong 
deterrent  to  contractor  ingenuity. 

If  both  the  Customer  and  the  Contractor  agree,  and 
I'm  sure  we  do,  that  tailoring  specifications  to 
the  particular  needs  of  a given  program  is  cost 
effective,  then  tailoring  should  be  accomplished. 
When  it  should  be  accomplished,  by  who,  when,  and 
how  is  important.  We  believe  the  quality  specifi- 
cations should  be  tailored  by  the  Customer  and 
made  a part  of  the  RFP,  We  further  believe  that 
the  RFP  should  also  encourage  additional  sugges- 
tions for  tailoring  by  the  contractor  without  fear 

{real  or  imagined)  of  being  found  nonresponsive. 

fter  contract  award,  contract  incentives  should  be 
given  for  continued  tailoring  through  change  pro- 
posals or  some  other  means. 

The  Contractor  should  have  the  option  to  respond  to 
an  RFP  with  a detailed  explanation  of  the  methods 
by  which  he  plans  to  carry  out  the  specified 
requirements.  The  mechanism  for  this  is  the  prep- 
aration of  a "Program  Plan."  We  recommend  that 
"Plans"  be  required,  approved  and  made  contractual 
by  the  customer,  and  thereafter  used  as  the  basis 
for  judging  contractual  compliance. 

Engineering  Hardware  Phase 

The  purpose  of  quality  assurance  during  research 
and  development  (engineerinn  and  prototype  hard- 
ware) is  to  enhance  product  acceptability  and  to 
prepare  for  product  acceptance.  This  is  basically 
accomplished  through  quality  assurance  participa- 
tion in  design  reviews  and  through  quality  assur- 
ance planning  for  subsequent  flight  hardware 
procurement  and  production. 

The  essential  role  of  Product  Assurance  in  the 
design  process  is  to  ensure  inspectability  and 
testability.  We  believe  this  can  best  be  accom- 
plished in  the  program  area  by  an  Integrated  team 
comprised  of  Program  Office,  Engineering,  Manufac- 
turing, Product  Assurance  and  Involved  support 
functions. 


We  believe  we  should  be  a "help-mate"  to  Engineering 
during  the  engineering/prototype  hardware  production 
phase  by  assisting  "as  required."  It  must  be 
stressed,  however,  that  the  hardware  in  this  phase 
is  not  intended  to  be  used  as  a deliverable  flight 
item  or  for  testing  of  flight  hardware.  Our  main 
emphasis  is  workmanship  inspection  as  opposed  to 
Corrective  Action.  Discrepancy  documentation  and 
corrective  action  should  be  without  AFQA  or  OCAS 
participation. 

Flight  Hardware  Phase 

When  the  Program  evolves  from  the  Engineering  hard- 
ware phase  to  qualification/flight  hardware,  Q.A. 
involvement  must  be  expanded.  During  the  Design 
phase,  Product  Assurance  and  Manufacturing  should 
sign  the  design  review  package  at  both  PDR  and  CDR 
and  all  subsequent  changes. 

We  believe  that  the  quality  requirements  we  impose 
on  suppliers  should  be  tailored.  For  example,  the 
requirements  imposed  on  ho-hum  procurements  would 
be  deferent  than  for  "critical"  items.  Every 
attempt  is  made  to  use  our  Supplier's  existing 
quality  system  and  procedures  to  eliminate  the 
necessity  for  him  to  "re-invent"  the  wheel  to  com- 
ply with  our  standards.  Perhaps  we  can  forego 
having  the  supplier  prepare  a special  Q.A.  Program 
Plan  if  a standard  plan  is  available  and  if  it  is 
adequate. 

Perhaps  the  major  phase  is  in  the  in-house  manufac- 
turing and  assembly  activity.  We  feel  all  produc- 
tion work  must  be  performed  under  controlled 
conditions  to  Engineering  requirements  using 
Manufacturing  Work  Instructions  (Shop  Orders,  etc.) 
as  controls. 

Another  major  effort  is  "Test  and  Delivery."  Pro- 
duct Assurance  must  ensure  that  final  inspection 
and  test  of  completed  items  are  performed  to  give 
an  overall  measure  of  each  item's  conformance  to 
end  product  specification  requirements. 

It  must  be  stressed  that  a vigorous  program  to 
detect  the  real  cause  of  all  failures  must  be  pur- 
sued so  that  corrective  action  can  be  taken  to 
prevent  recurrence  of  similar  discrepant  conditions 
if  only  on  other  similar  programs. 

In  conclusion,  I believe  for  this  type  of  program 
that  MIL-Q-9858A  should  be  invoked  but  that  certain 
features  should  be  tailored  out;  for  example.  Cost 
of  Qual ity. 

Deviation  from  the  "norm"  can  be  expensive  and 
possibly  counterproductive. 
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WORKMANSHIP  IMPROVEMENT  AND  STANDARDIZATION 


Presented  by  L.  Katz  in 
The  Aerospace  Corp. 


As  an  attempt  at  improving  and  standardizing 
workmanship  levels  of  operator  performance  on  high 
reliability  programs  several  suggestions  were  made: 

1.  A workmanship  manual  be  prepared  by  a 
government  agency.  This  manual  should 
provide  contractually  imposed  standards 
of  workmanship  quality.  It  should  not  be 
limited  to  just  soldering,  but  should 
cover  all  standard  workmanship  items 
associated  with  high  reliability  types  of 
electronic  manufacturing. 

2.  Schools,  established  by  a government 
agency,  should  train  and  certify  both 
government  and  industry  instructors. 

These  classes  should  include  both  the 
theory  behind  quality  judgments  and  the 
practice  and  training  of  performing 
quality.  The  workmanship  manual  pre- 
viously addressed  could  be  used  as  part 
of  the  tutorial  text  material. 

3.  Certified  operator  training  programs, 
using  instructors  that  have  been  certi- 
fied by  the  government  schools,  be  estab- 
lished at  various  government  and  industry 
locations.  These  programs  would  select 
candidates  who  qualify  for  high  reliabil- 
ity work  based  on  dexterity,  aptitude  and 
other  psycho  technical  testing  and  then 
teach  them.  Not  all  taking  the  course 
need  necessarily  pass  it.  Those  who  do 
pass  should  be  accredited  and  periodically 
re-accredited.  Graduates  of  these  pro- 
grams would  qualify  for  operator  or 
inspector  classifications  and 
certification. 

4.  Continuing  grading  should  be  established 
for  operators  and  inspectors.  This 
grading  would  be  a contractual  commitment 
that  evaluates  the  quality  performance  of 
each  individual  based  on  actual  rejections 
by  inspection  and  quality  department 
verifications  of  inspection  thoroughness. 

With  the  above  four  items  implemented,  the  level  of 
workmanship  quality  should  improve  and  should 
approach  a single  standard  of  excellence  for  all  of 
the  high  reliability  electronics  industry. 


THE  AFCMD  CONTRACTOR  MANAGEMENT  SYSTEM 
EVALUATION  PROGRAM 

Edward  E.  Mazzanti 
Chief,  Quality  Assurance  Division 
AFPRO/TRW 


This  portion  of  the  workshop  concerns  government 
in-plant  involvement  at  contractor's  facilities. 

The  purpose  of  this  briefing  is  to  give  you  some 
insight  as  to  what  this  involvement  entails  for 
those  contractors  under  the  cognizance  of  AFCMD  — 
The  Air  Force  Contract  Management  Division.  Most 
of  our  involvement  centers  around  our  Contractor 
Management  System  Evaluation  Program,  or  CMSEP,  and 
that  is  the  subject  of  my  briefing  today. 

Why  CMSEP?  Well,  the  answer  is  very  simple.  Our 
old  techniques  were  considered  ineffective.  In  the 
late  60' s and  early  70's,  a series  of  surveys  were 
conducted  at  various  contractors  facilities.  Time 
and  again  the  same  deficiencies  in  contractor  oper- 
ations were  reported.  During  the  same  period  some 
serious  hardware  deficiencies  on  major  Air  Force 
programs  were  noted  and  this  caused  some  serious 
concern  throughout  AFCMD.  So  it  was  determined 
that  a new  approach  was  needed.  Something  that 
would  emphasize  prevention  of  system  deficiencies 
that  could  in  turn  cause  hardware  defects.  It  had 
to  be  conducted  within  available  AFCMD  resources  - 
which  meant  that  we  had  to  change  our  way  of  doing 
business  and  redirect  our  efforts  toward  prevention 
and  it  was  decided  the  way  to  do  it  was  through 
evaluation  of  Management  Systems. 

This  concept  is  based  on  the  principle  that  product 
quality  is  the  direct  result  of  management  quality. 
It  is  also  based  on  the  assumption  that  a responsi- 
ble contractor  dedicated  to  the  delivery  of  a qual- 
ity product  within  cost  and  schedule  will  develop 
management  systems  in  an  orderly  and  planned  manner. 
It  then  makes  sense  that  we  in  AFCMD  expend  a major 
portion  of  our  efforts  toward  ensuring  the  existence 
and  operation  of  adequate  contractor  management 
system. 

In  the  fall  of  1973,  under  the  direction  of  General 
Nunn,  the  AFCMD  Contractor  Management  System  Evalu- 
ation Program  was  established.  Management  System 
Indicators  (MSIs)  were  developed  as  the  primary 
component  of  the  program  and  these  are  used  for 
guiding  Air  Force  Plan  Representative  Offices 
(AFPROs)  evaluation  of  management  systems. 

Under  the  program,  each  functional  area  of  an  AFPRO 
is  responsible  for  full  evaluation  of  its  contractor 
counterpart  organization. 

Conmand  Management  is  the  responsibility  of  the  Air 
Force  Plant  Representative  and  he  must  ensure  eval- 
uation of  his  counterpart  who  is  the  contractor's 
chief  operating  official. 

There  are  a number  of  MSIs  assigned  to  each  func- 
tional area  and  each  is  a question  directed  at  a 
discrete  portion  of  the  contractor's  management 
system  addressed  by  the  functional  area. 

The  MSIs  provide  an  evaluation  approach  that  will 
address  all  levels  of  management  within  a contrac- 
tor's organization.  The  Conmand  Management  MSIs 
(CM-1  & 2)  were  developed  to  ensure  that  the  AFPR 


policies  and  his  assignments  of  authority  and 
responsibilities  to  top  executive  and  functional 
managers.  The  first  MSI  in  each  of  the  other  func- 
tional areas  provides  an  evaluation  similar  to  CM  1 
& 2 except  it  is  conducted  at  a lower  level  in  the 
organization.  For  example:  The  first  MSI  from 
Quality  Assurance  which  is  designated  as  QA-1  will 
be  used  by  the  AFPRO  Quality  Assurance  Division  Chief 
to  ensure  that  his  contractor's  counterpart,  the 
functional  manager  of  Quality  Assurance,  has  properly 
stated  his  policies  and  assignments  of  authority  and 
responsibilities  to  fulfill  his  function.  Finally, 
all  other  MSIs  are  used  to  determine  the  degree  of 
implementation  of  authority  and  responsibilities  at 
lower  levels  of  management  in  all  areas. 

For  example,  one  of  the  15  MSIs  for  Quality  Assur- 
ance, QA-3,  addresses  that  portion  of  the  contrac- 
tor's Quality  Assurance  Management  System  that 
concerns  work  instructions.  One  of  the  9 for 
Manufacturing  Operation,  PD-9,  addresses  that  por- 
tion of  the  contractor's  manufacturing  management 
system  that  concerns  packaging,  handling  and 
transportabil ity. 

The  MSI's  are  rather  broad  and  don't  really  lend 
themselves  to  a detailed  evaluation  — but  they're 
not  intended  to. 

Instead,  there  are  a series  of  Condition  Questions 
assigned  to  each  MSI.  These  are  probing  questions 
that  assist  an  evaluator  in  investigating  the  status 
of  an  element  of  the  area  covered  by  the  MSI.  There 
are  twelve  Condition  Questions  that  address  the 
2 MSIs  for  Command  Management  — 92  for  the  15  MSIs 
for  Quality  Assurance  and  so  on.  Typically  QA-3, 
the  MSI  dealing  with  work  instructions,  contains 
five  condition  questions. 

As  you  can  see,  these  questions  are  more  specific 
and  this  is  where  the  evaluations  are,  in  fact, 
conducted.  With  that,  let's  take  a look  at  the 
evaluation  process. 

This  process  is  actually  a test  for  existence, 
adequacy  and  compliance.  Existence  of  contractor 
procedures  that  address  the  management  system  in 
question.  The  next  step,  is  to  determine  if  the 
procedures  adequately  meet  all  requirements  or 
standards.  And  finally,  the  compliance  check  veri- 
fies that  the  contractor  is  in  fact  complying  with 
his  own  procedures. 

To  assist  in  this  evaluation  process,  further  cri- 
teria are  developed  for  each  Condition  Question. 

There  are  Assessment  Criteria  which  are  questions 
or  short  statements  developed  to  test  existence  and 
adequacy  of  the  documented  system.  Assessment 
Criteria  are  developed  from  such  sources  as  MIL- 
STDs,  ASPR  clauses,  regulations,  industry  handbooks 
or  other  documents  describing  various  features  of 
an  acceptable  management  system. 

Compliance  Criteria  are  also  developed  which  relate 
directly  to  Condition  Questions  and  these  are 
extracted  from  the  Contractor's  Written  Procedures 
and  are  used  to  determine  if  the  contractor  person- 
nel are  complying  with  their  own  documented 
requirements. 


THE  DLA  CONTRACTOR  ASSESSMENT  PROGRAM 


Spencer  Hutchens 

Dep.  Dir.,  Quality  Assurance  Directorate 
DCASR  - LA 


The  Contractor  Assessment  Program,  within  the 
Department  of  Defense  is  a program  to  reduce 
involvement  of  Government  personnel  in  the  day-to- 
day  operations  of  selected  contractors,  while,  at 
the  same  time,  assuring  that  quality  of  product  is 
maintained.  The  paper  reviews  development  of  the 
program  since  its  conception  by  the  DoD  Quality 
Assurance  Council  in  early  1975.  By  1976,  the 
detailed  implementation  plan  developed  by  Defense 
Logistics  Agency  had  been  approved  and  a test  pro- 
gram initiated. 

Contractor  eligibility  criteria  are  discussed. 

These  include  the  requisite  that  the  contractor  be 
a DoD  prime  contractor,  with  MIL-Q-9858A  require- 
ments and  that  the  item  in  question  shall  have  been 
in  continuous  production  for  a reasonable  period  of 
time.  Additionally,  the  contractors'  quality 
management  must  be  considered  capable  of  continuing 
to  maintain  quality  production,  the  contractors' 
pre-award  and  post-award  history  must  be  good, 
there  should  be  no  major  quality  deficiencies  and 
one  or  less  Materiel  Deficiency  Report  issued  per 
million  dollars  shipped  of  product  shipped,  among 
other  criteria. 

Contractors  initially  proposed  by  the  DCAS  Regions 
for  inclusion  in  the  program  are  screened  and 
evaluated  by  a Contractor  Assessment  Advisory 
Board,  a sub-committee  of  the  DoD  Quality  Assurance 
Council.  Subsequent  to  this  review,  coordination 
with  the  buying  agencies  is  effected.  An  agreement 
covering  the  contractors'  participation  in  the 
program  - on  a volutary  basis  and  with  no  increase 
in  his  costs  - is  signed.  The  Government  develops 
a Quality  Assurance  plan  tailored  to  the  specific 
situation  at  each  contractors'  plant. 

The  Phase  I Trial  Program,  initiated  in  late  1976, 
included  RCA  and  Bendix.  One  product  line  at  Texas 
Instruments  subsequently  was  added.  Cleveland 
Pneumatic  was  selected  for  participation  in  early 
1978. 

Evaluation  of  the  Phase  I Trial  Program  was  encour- 
aging, and  indicated  that  objectives  of  the  program 
were  being  met  with  no  degradation  of  product 
quality.  Under  a Phase  II  Test  Program,  developed 
in  early  1978,  six  companies  are  being  considered 
for  inclusion. 


To  assure  mission  success,  it  is  necessary  that  the 
Manufacturing  and  Inspection  instructions  faith- 
fully reproduce  the  Engineering  design.  Engineer- 
ing review  and  approval  of  those  work  instructions 
has  been  found  to  be  a good  technique  to  help  assure 
the  effectiveness  of  the  work  instructions.  Let  me 
summarize:  The  critical  areas  or  operations 
required  for  mission  success  must  be  identified. 

The  associated  work  instructions  must  be  specific, 
not  general,  and  Engineering  should  review  and 
approve  the  critical  work  instructions  to  verify 
that  they  will  reproduce  what  Engineering  designed. 

Work  instructions  are  those  documents  that  are  used 
by  Manufacturing,  Inspection,  and  the  Test  groups 
to  build  and  test  a product.  It  is  with  these 
work  instructions  that  the  Engineering  design  is 
converted  from  paper  to  hardware  or  operating  soft- 
ware. Questions  usually  arise:  Does  the  end 
product  faithfully  represent  Engineering's  designs? 
Did  the  Manufacturing,  Inspection,  and  Test  opera- 
tions create  the  product  as  directed  by  the 
Engineering  design?  To  eliminate  these  questions, 
the  work  effort  must  be  well  defined.  "What  is 
critical"  must  be  identified;  "when  to  inspect" 
must  be  determined;  "how  to  inspect"  must  be  speci- 
fied; as  well  as  determining  "which  test"  should 
be  run.  This  is  the  dilemma  that  Manufacturing, 
Inspection,  and  Test  groups  face.  Without  Engineer- 
ing's help,  mission  success  is  surely  put  in  doubt. 
The  engineer  knows  more  about  what  is  critical  with 
his  designs  than  the  other  functions  could  possibly 
know  the  other  functions'  expertise,  however, 
knows  "how  to  inspect,"  "how  to  test,"  and  can 
determine  "when  to  inspect"  and  "when  to  test." 

But  to  determine  "what  is  critical"  and  how  it 
affects  the  mission,  is  best  done  by  Engineering. 

Let  me  take  you  briefly  through  an  example  of  such 
a requirement  that  was  critical  in  order  to  assure 
mission  success.  Engineering  played  the  basic  key 
role.  A recent  contract  requirement  required  the 
vehicle  to  be  designed  using  fractjre  mechanics 
criteria.  Parts  that  were  considered  fracture 
critical  had  to  be  designed  to  withstand  initial 
flaw  size  of  0.050"  depth,  located  in  the  most 
critical  portion  of  that  part  for  the  total  life 
of  the  vehicle  without  becoming  critical.  Engineer- 
ing identified,  on  the  drawings,  which  parts  were 
fracture  critical.  They  also  identified  the  crit- 
ical areas  on  each  of  those  parts  and  the  direction 
of  the  critical  stresses.  This  information  allowed 
Quality  Assurance  to  identify  the  proper  inspection 
processes  to  be  used.  The  Inspection  requirements 
document  was  also  specifiti  on  the  drawing.  Hence, 
by  the  drawing  specifying  the  various  critical 
aspects  of  the  part  and  calling  out  the  Inspection 
requirements,  the  inspector  is  led  to  the  Specific 
Inspection  Instruction  by  part  number  and,  conse- 
quently, to  the  Inspection  Instruction  how-to's  to 
do  the  job  properly.  It  was  through  this  technique 
we  were  able  to  get  the  high  reliability  necessary 
to  employ  fracture  critical  design  using  a small, 
critical  flaw  size.  Each  of  these  Inspection 
Instructions  are  processed  through  Engineering  for 
approval.  Review  by  Engineering  provides  a check 
and  balance  on  the  work  Instruction  system.  It 


makes  sure  that  the  instructions  cover  the  critical 
items  to  assure  mission  success,  and  it  also  helps 
insure  that  the  build  and  inspection  sequence  is 
realistic.  When  we  had  the  engineer's  approval,  we 
knew  we  had  understood  his  design. 

A major  benefit  that  occurred  by  having  Engineering 
involved  in  the  work  instruction,  was  their  obtain- 
ing a better  appreciation  of  the  needs  of  Quality 
Assurance  and  the  value  of  Quality  Assurance's  inputs 
to  the  design  process.  Engineering  now  understands 
Inspection's  needs  and  readily  accepts  our  comments 
on  their  design.  Many  designs,  at  Engineering's 
request,  are  now  reviewed  by  Quality  Assurance  in 
order  to  assure  that  the  part  to  be  fabricated  is 
inspectable.  With  the  processes  that  are  available 
today,  parts  can  be  designed  and  built  that  cannot 
be  reliably  inspected  — a sure  way  of  not  insuring 
mission  success.  Another  side  benefit  is  the  fact 
that  it  forces  another  look  at  the  design.  By  rid- 
ding designs  and  work  instructions  of  error  early  in 
the  program  cycle,  more  attention  can  be  given  to 
properly  reproducing  the  products  and,  hence,  assure 
mission  success. 

In  summary,  I would  like  to  again  point  out  that  the 
critical  areas  for  mission  success  must  be  identi- 
fied. The  work  instructions  must  be  specific  to  the 
needs  and  not  general.  To  assure  that  they  all 
create  the  proper  product,  Engineering  must  review 
and  approve  those  work  instructions. 
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PRODUCTION  FOR  SPACE  AND  ITS  ASSOCIATED  PROBLEMS 

John  P.  Flanagan 
Production  Section  Manager 
Honeywell  Inc. 

The  American  assault  on  space  began  in  the  late 
1950' s.  Success  came  for  Explorer  1 on  January  31, 
1958,  when  a 70  foot  tall  Jupiter  Rocket  succeeded 
in  putting  a 30  pound  payload  into  orbit.  This  was 
the  beginning  of  our  space  age.  Since  then  the 
space  program  has  seen  some  failures  but  many,  many 
successes.  The  360  foot  Saturn  5 put  a 200,000 
pound  sky  lab  into  orbit  and  pushed  manned  space- 
craft to  the  moon.  The  Mercury,  Gemini  and  Apollo 
manned  flight  programs  were  the  ultimate' in  well 
publicized  success. 

But,  this  publicized  success  has  hidden  from  public 
view  the  problems  that  have  plagued  the  manufac- 
turers of  space  hardware.  With  the  early  restric- 
tions of  available  booster  power  came  constraints 
on  weight,  power  and  precious  volume.  These  con- 
straints led  to  cramming  more  and  more  piece  parts 
and  circuitry  onto  the  valuable  real  estate  of 
printed  circuit,  boards.  The  boards  themselves 
became  20  or  more  layers  to  accommodate  all  of  the 
required  interconnections.  This  has  led  to  minia- 
turization of  piece  parts  and  the  problems  with 
handling  such  minute  items  during  the  assembly  pro- 
cess. Today's  plated  wire  memories  have  2 mil 
wires  which  are  inserted  into  4 mil  tunnels  and 
soldered  to  landing  pads  which  are  on  10  mil 
centers.  Just  as  state-of-the-art  advancements  are 
being  made  in  hardware  design,  so  too  must  produc- 
tion methods  and  processes  be  improved.  Microscopes 
of  higher  and  higher  power  are  now  employed  ,iust  to 
be  able  to  see  what  one  is  doing.  More  and  better 
lighting  and  lenses  are  being  employed  for  assembly 
and  for  inspection.  Gloves  and  finger  cots  are 
commonplace  annoyances  for  Production  personnel. 
Operators  working  with  static  sensitive  parts  must 
wear  special  smocks  and  grounded  wrist  straps,  and 
work  on  special  tabletops. 

The  problem  of  heat  dissipation  has  taken  a few  new 
twists.  Forced  air  cooling  is  becoming  a thing  of 
the  past  giving  way  to  conductive  dissipation  of 
heat  through  use  of  thermal  overlays.  The  overlay 
further  complicates  the  process  of  soldering  piece 
parts  to  circuit  boards.  The  soldering  operation 
must  be  perfectly  executed  at  the  bottom  of  a 
"well."  The  use  of  aluminum  foil  wrapped  around 
stiff  bars  for  further  heat  dissipation  is  another 
engineering  innovation  which  the  assembly  operators 
would  rather  not  have. 

The  use  of  computer  controlled  automatic  test  equip- 
ment which  minimizes  dependence  on  human  operators 
is  becoming  more  and  more  a requirement.  Diagnos- 
tic software  which  pinpoints  failures  is  a neces- 
sity. This  Is  necessary,  as  the  environment  to 
which  the  hardware  is  subjected  in  flight  must  be 
more  than  matched  during  the  test  process.  Opera- 
tion over  temperatures  from  -85°F  to  +200°F  are  not 
unique.  Vibration,  shock,  vacuum,  salt  water  spray, 
100*  relative  humidity,  dust  and  accelerations  are 
environments  that  the  hardware  must  survive. 

Survivabil Ity  is  a by-word  in  our  Industry.  Some 
hardware  must  sjrvive  the  radiation  effects  from  a 
nuclear  explosion  or  from  passing  through  the  Van 
Allen  radiation  belt  and  it  still  must  perform  its 
intended  mission  flawlessly.  The  Viking  Lander, 


went  through  a "cooking  process"  at  233°F  for 
156  hours  in  order  to  sterilize  this  earth  built 
hardware  so  that  the  planet  Mars  would  not  be  con- 
taminated by  the  earthman's  visit  and  mission  per- 
formance. The  spacecraft  must  survive  and  the 
planet  must  survive. 

As  we  reap  more  benefits  from  space  missions,  we 
generate  demands  for  more  missions.  Our  boosters 
have  become  more  powerful,  and  yet  the  payloads 
have  become  even  more  sophisticated.  With  sophisti- 
cation comes  higher  cost  and  as  the  payload  becomes 
more  expensive,  the  need  to  assure  mission  success 
becomes  more  of  a requirement.  The  ultimate  in 
payload  sophistication,  of  course,  is  man  himself. 

It  was  man  in  space  who  applied  the  fix  to  the 
crippled  sky  lab.  However,  many  of  these  sophisti- 
cated payloads  form  the  backbone  of  our  nation's 
defense  capability.  To  many,  this  is  by  far  the 
most  important  reason  that  demands  flawless,  suc- 
cessful mission  performance  regardless  of  cost  or 
hardship.  To  improve  the  probability  of  mission 
success,  we  have  upgraded  our  piece  parts  and 
established  parts  reliability  requirements  unheard 
of  a decade  ago.  We  require  that  parts  be  trace- 
able from  cradle  to  grave.  Certified  parts  must  be 
installed  by  certified  operators  with  certified 
instructions  and  certified  equipment.  This  has 
created  data  collection  systems,  piece  part  and 
paperwork  traceability  systems  and  configuration 
identification  systems  that  only  the  technology  of 
modern  computers  can  handle. 

This  leads  me  to  the  most  consistent  of  the  manu- 
facturer's problems  — that  of  soaring  costs.  We, 
in  manufacturing,  must  constantly  strive  to  reduce 
costs  while  complying  with  all  of  the  requirements 
for  reliability  and  at  the  same  time,  meeting  the 
proposed  delivery  schedule.  We  are  under  pressure 
to  utilize  more  sophisticated  equipment  in  the 
manufacturing  process  to  hold  or  reduce  the  cost 
line,  while  at  the  same  time  we  are  told  to  reduce 
energy  consumption.  Rework  to  correct  cosmetic 
blemishes  possibly  deteriorates  rather  than  enhances 
the  reliability  of  the  product.  We  must  constantly 
strive  not  to  lose  sight  of  the  fact  that  equipment 
performance  rather  than  beauty  is  the  prime 
requisite. 

Different  agencies  may  procure  the  booster,  the 
guidance  system,  the  computer,  the  payload  experi- 
ments, etc.,  for  the  same  mission.  Each  agency's 
particular  parts  specification  is  interpreted  by 
each  prime  contractor,  is  interpreted  again  by  all 
of  the  sub-contractors,  and  results  in  the  parts 
vendor  having  many  different  requirements  imposed 
on  him  for  the  same  basic  part  for  one  mission. 
Standardization  of  parts  specifications  for  all 
Government  Procuring  Agencies  would  go  far  to  help 
reduce  parts  costs  and  delivery  schedules,  and 
relative  at  least  one  source  of  the  manufacturer's 
problems. 

The  end  result  of  our  frustrations  and  hard  work  is 
the  reward  to  our  pride  when  missions  are  success- 
ful. We  always  want  to  identify  with  success.  We 
dread  the  consequences  of  our  hardware  failure  on 
our  nation's  position  at  the  world's  peace  tables, 
and  even  more  so,  on  the  human  payload  that  some 
missions  require.  Who  will  ever  forget  the  plight 
of  Apollo  13  when  its  April  1970  rendezvous  with 
the  moon  was  aborted  due  to  a spacecraft  accident. 
The  whole  world  was  caught  up  in  the  emotions  of 
the  moment  as  the  lives  of  those  three  astronauts 
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depended  upon  their  remaining  equipment  functioning 
outside  of  the  capacity  for  which  it  was  initially 
intended.  No  space  hardware  supplier  wants  such  an 
awesome  spectre  to  haunt  him  because  of  failure  of 
his  product.  Instead,  we  will  do  anything  to  over- 
come the  problems  and  hardships.  We  will  work 
within  all  constraints  while  trying  to  use  logic 
and  reason  to  remove  those  which  we  are  convinced 
are  unreasonable.  Whatever  it  takes,  MISSION 
SUCCESS  is  our  committment.  Our  reputations  demand 
it;  our  paychecks  depend  on  it;  our  survival 
requires  it. 


WORKSHOP  SUMMARY 
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SYNOPSIS  OF  ROUND  TABLE  DISCUSSIONS 

1 . Critical  Item  Control: 

• Problem 

Resources  allocated  to  monitoring  critical 
item  control  are  spread  too  thin. 

Cause 

Contractural  Requirements,  e.g.,  MIL-STD-1543, 
define  critical  items  too  broadly. 

Reommendation 

Revise  MIL-STD-1543  to  categorize  critical 
items  by  potential  effect  on  Mission  Success. 

Action 

SAMSO/ Industry  Team 

• Problem 

Quality  Management  has  failed  to  take  full 
advantage  of  available  analytical  techniques, 
e.g.,  FMEA  and  System  Safety  analysis  in  the 
planning  and  conduct  of  audits,  reviews  and 
inspection. 

Cause 

Lack  of  Management  attention  or  emphasis. 
Recommendation 

Make  Management  at  all  levels  aware  of  poten- 
tial benefits  of  using  these  techniques. 

Action 

Industry 

2.  Establishing  Realistic  QA/Mfg  Requirements: 

• Problem 

Establishing  realistic  QA/Mfg  Requirements  for 
R&D  and  low  volume  programs. 

Cause 

Blanket  imposition  of  specifications. 
Recommendations 

Encourage  realistic  tailoring  by  SPO.  Encour- 
age use  of  contractor  prepared  plans  to  tailor 
requirements.  (Acceptable  to  Government  CAO 
& SPO)  (Place  on  contract) 

Action 

SAMSO 

3.  Workmanship  Standards  and  Practices 

• Problem 

Lack  of  workmanship  standardization  within  the 
industry. 

Cause 

NO  organized  direction  by  Government  or 
Industry. 


Recommendations 

Government  contract  for  identifying  and 
preparing  minimum  workmanship  acceptance 
criteria  - OR  - Professional  society  (i.e.,: 

AIA,  ASQC,  etc.)  to  recommend  and  prepare 
minimum  workmanship  acceptance  criteria. 

Action 

Government  or  Industry  Association. 

• Problem 

implementation  of  workmanship  standards  for 
"Hi-Rel"  applications. 

Causes 

Cost.  - Resistance  to  change.  - lack  of  top 
management  involvement. 

Recommendation 

Specific  contractural  imposition  of  standards. 

Action 

Government 

• Problem 

Operator/Inspector  training  for  "HI-Rel". 
parts  fabrication. 

Cause 

Cost.  — Lack  of  standard  requirements.  — No 
organized  direction  by  Government  or  industry. 

Recommendations 

Workmanship  incentive  at  operator  level.  — 
Establish  universal  training  requirements.  — 
Indoctrination  of  top  management  to  cost 
effectiveness  of  improved  training. 

Action 

Industry. 

4.  Government/Industry  Mission  Assurance  Interface: 

• Problems 

Poor  tailoring  of  contract  requirements.  — 
Inconsistent/unrealistic  letters  of  delegation. 
Insufficient  coordination  between  SPO,  CAO  and 
contractor  QA. 

Cause 

Lack  of  qualified  QA  personnel  at  Buying 
Offices  (SPO) 

Recommendation 

Establish  a program  for  assuring  qualified  QA 
personnel  are  assigned  to  SPO's 

Action 

SAMSO 

5.  Transition  From  Design  to  Manufacturing: 

• Problem 

Mission  critical  areas  do  not  receive  appro- 
priate attention. 
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Cause 


Existing  “critical  item"  clauses  require 
listings  of  critical  items  without  sufficient 
follow-thru,  e.g.,  identification  on  engineer- 
ing drawings. 

Recommendation 

RFP  should  specifically  require  that  critical 
characteristics  be  identified  on  engineering 
drawings. 

Action 

SAMSO/ Industry  Team 

• Problem 

Work  instructions  may  not  cite  acceptance.  - 
Criteria  for  critical  items  due  to  lack  of 
involvement  by  design  engineering. 

Cause 

Lack  of  design  engineering  involvement  in 
development  of  work  instructions. 

Recommendation 

Press  for  review  and  sign-off  on  work.  — 
Instructions  by  design  engineering  on  all 
critical  items. 

Action 

Industry. 

Problems  of  Low  Volume  Production: 

• Problem 

High  cost  and  schedule  impacts  are  associated 
with  low  volume  production. 

Causes 

Low  volume  precludes  economies  such  as  lot  or 
batch  build.  - Fixed  costs  exist  regardless  of 
quantity  being  produced.  - Proprietary 
processes  at  vendors  or  long  lead  items 
frequently  cause  program  delays. 

Recommendations 

Encourage  innovation  by  incentive  clauses.  - 
Tailor  existing  MGMT/Production  Systems  (DWG. 
Release,  detailed  work  instruction  etc.)  for 
small  programs.  - Assure  schedule  realism  at 
time  of  RFQ  by  Government.  - Buy  proprietary 
process  rights  to  avoid  schedule  impact. 

Action 

SAMSO/ Indus try 

• Problem 

Lack  of  viable  supplier  response  to  low  volume 
procurements. 

Cause 

Too  many  MIL-SPEC  requirements  for  suppliers 
to  be  bothered  with.  Too  much  investment  for 
small  amount  of  profit. 

Recommendations 


on  suppliers,  which  avoids  waiver/deviation 
methods. 

Action 

Industry. 

• Problem 

Electrical  connectors  are  plagued  with  chronic 
quality  and  schedule  problems. 

Cause 

Unknown 

Recommendation 

An  industry  or  Government  organization  or 
Select  Committee  should  examine  this  issue. 

Action 

Government  or  Industry. 


Overbuy  parts  and  screen  in-house.  - Establish 
method  of  tailoring  MIL-SPECS  for  imposition 
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Summary 


M1L-STD-1540,  TEST  REQUIREMENTS  FOR  SPACE 
VEHICLES  STATUS  AND  FUTURE  PLANS 
(SYNOPSIS) 

Lt.  Col . Duane  Baker 
SAMSO/AWST 


An  effective  test  and  evaluation  program  is  a 
mandatory  requirement  to  assure  mission  success  of 
our  military  spacecraft.  To  emphasize  the  need  for 
adequate  testing,  examples  of  system  failures  are 
discussed  which  may  have  been  prevented  by  more 
effective  testing. 

Due  to  the  unique  characteristics  of  these  systems, 
a special  Military  Standard  was  developed  to 
address  the  testing  required  to  effectively  simu- 
late the  expected  environment  and  uncover  deficien- 
cies before  the  systems  are  put  into  operation. 
MIL-STD-1540A  was  issued  in  1974  and  considerable 
experience  has  been  gained  with  the  use  of  this 
document.  In  this  discussion  the  current  require- 
ments in  MIL-STU-1 540A  are  outlined  with  some  of 
the  background  and  rationale  behind  these  require- 
ments. Experience  has  shown  some  deficient  areas. 
These  are  discussed  along  with  current  plans  for 
revising  the  document. 


IMPLEMENTING  1540  ON  GPS  (SYNOPSIS) 

Mr.  Martin  L.  Adams 
RI  Space  Div. 

Mgr.  Systems  Effectiveness 
Rockwell  International/DS 


GPS  was  the  first  satellite  program  to  have  MIL- 
STD  1540  fully  imposed.  Experts  wrote  the  MIL- 
SPEC  but  non-experts  must  make  its  implementation 
a practical  reality.  Although  GPS  is  considered  by 
SAMSO  to  have  achieved  the  highest  level  of  compli- 
ance, it  still  took  over  70  meetings  of  the  Test 
Criteria  Review  Board,  150  waivers  and  X test  cri- 
teria changes  notices.  This  means  we  struggled  to 
change  or  accepted  some  degree  of  non-compliance  - 
What  changes  and  why. 

1 ) Sequence  Changes 

Acceptance  and  qual  environments  were  run  con- 
secutively. Should  we  have  run  the  full  ATP, 
then  gone  into  qual? 

2)  Tolerances 

Many  waivers  were  approved  because  we  couldn't 
hold  the  +3  db  amplitude  for  power  spectral 
density  during  vibration.  Should  1540  be 
revised  to  6 db? 

3)  Burn-in 

Literal  compliance  w/1540  will  cause  37.5  temp, 
cycles  on  a box  with  an  85°C  temp,  swing  to 
achieve  the  required  300  hours  of  burn-in. 

Test  time  during  environments  is  expensive. 

Is  this  requirement  necessary? 

4)  Equitable  Retest 

Were  you  ever  asked  to  perform  a repeat  of  test 
sequences  which  you  thought  were  unreasonable, 
after  you  had  a failure?  What  checklist  could 
you  prepare  to  decide  an  equitable  retest, 
after  failure  or  interruption? 


INTERFACE  TESTING  CONSIDERATION  (SYNOPSIS) 

Mr.  Nick  G.  Babich 
The  Boeing  Co. 

Testing  of  interfaces,  both  on  the  subsystem  and 
system  level,  has  always  been  a primary  discipline 
in  the  sequence  of  verifying  functional  compatibil- 
ity of  booster  and  payload/spacecraft  systems. 

Many  sources  of  documentation,  Mil  Std  1540A 
included,  try  to  define  a logical,  thorough  set  of 
test  requirements  that  will  assure  compatible  sys- 
tems and  minimize  the  risks  associated  with  high 
technology,  often  state-of-the-art,  designs.  How- 
ever, the  effectiveness  of  Mil  Std's  and  other  test 
documentation  can  be  significantly  influenced  by 
the  programmatic  test  philosophy  existing  at  the 
time  of  application.  The  often-used  cliche,  "Test 
what  you  fly  and  fly  what  you  test"  may  be  a “test 
goal”  but  often  ends  up,  due  to  physical  and/or 
practical  reality,  only  partially  attainable.  Thus, 
in  order  to  obtain  a thorough  “wringing  out"  of  all 
interface  parameters,  a total  system  concept  must 
be  kept  in  the  forefront. 

Some  considerations  for  discussion,  to  reevaluate 
current  interface  testing  practices  and  focus  atten- 
tion on  those  checks  and  balances  needed  to  assure 
successful  operational  performance  of  the  payload/ 
spacecraft  systems,  should  include  the  following: 

Use  of  a total  Systematic  Approach 

1.  Payload/Spacecraft  Systems  to  Booster 
Systems  (AVE,  GSE,  TSE). 

2.  Consideration  of  ALL  System  Interfaces 
(Electrical,  Mechanical,  Structural, 
Envelope,  Nuclear). 

Use  of  MIL  STD  Requirements  vs.  ICD  Parameters 

1.  Test  to  verify  compliance  with  Mil  Std's 
or  ICD' s? 

2.  Testing  to  first  active  element  or  beyond? 

Optimization  of  Test  vs.  Analysis 

1.  Design  philosophy  of  interface  system 
simulators. 

2.  Need  for  retest  after  completion  of  final 
design. 

3.  Need  for  interface  verification  on  each 
flight  unit. 

Test  Scheduling 

1.  Implementation  of  a Building  Block  Concept. 


ROLE  OF  DEVELOPMENT  TESTING  IN  COMPONENT/SYSTEM 
LEVEL  TEST  PROGRAM  (SYNOPSIS) 

Mr.  Dean  L.  Lindstrom 
Hughes  Aircraft  Co. 

1)  Schedule  Allocation  for  Development  Testing 

Most  space  programs  produce  so  few  flight  or 
development  models  that  they  seldom  reach  full 
design  maturity.  To  accelerate  this  process, 
development  testing  needs  to  include  test 
designed  to  uncover  weaknesses  as  well  as  to 
develop  capability.  Unfortunately,  this  part 
of  development  testing  is  often  over-looked. 

The  designer  hardly  has  time  to  develop  the 
design,  much  less  find  its  problems.  This  may 
be  as  much  his  fault  for  not  taking  time  to 
plan  as  it  is  the  Program  Manager's  for  not 
allowing  time  to  plan.  Time  should  be  sched- 
uled for  problem  prevention,  as  it  is  for 
problem  correction. 

2)  Simulation  of  On-orbit  Operation 

It  is  clear  that  many  failures  occur  on  orbit 
because  that  is  the  first  time  that  mode  was 
tested.  Better  simulation  of  operational  modes 
is  one  answer,  but  can  be  expensive.  However, 
even  simulation  of  operational  conditions  is 
not  adequate  to  eliminate  failures  that  are 
time  and  stress  dependent.  More  important  than 
operational  simulation  is  the  application  of 
sufficient  stress  to  accelerate  those  failures 
which  can  occur  in  unique  operational  modes  and 
under  time  and  stress  conditions.  The  test 
program  should  be  planned  so  that  development, 
qualification,  and  acceptance  tests  at  compo- 
nent and  system  level  are  adequate  to  uncover 
failure  modes  and  eliminate  potential  failures 
under  operational  conditions. 

3)  Failure  and  Test  Data  Analysis  and  Feedback 

Each  company  has  different  strengths  and  weak- 
nesses. Setting  up  test  programs  to  answer 
everyone's  weaknesses  is  very  expensive.  A key 
to  optimizing  your  test  program  is  to  assess 
the  problems  which  you  have  had.  Then  design 
a test  program  to  prevent  those  problems. 


IMPROVING  TEST  EFFECTIVENESS  (SYNOPSIS) 

Mr.  Bernard  J.  O'Brien 
TRW  Space  Systems 


This  discussion  takes  a critical  look  at  some  of 
the  considerations  which  influence  the  planning, 
implementation,  analysis  - and  hence  the  overall 
effectiveness  of  component  and  system  - level  test- 
ing for  space  systems.  It  includes  a discussion  of 
constraints  placed  on  innovation  and  flexibility, 
and  looks  at  some  measurements  of  screening 
effectiveness. 


WORKSHOP  SUMMARY 


TEST  WORKSHOP  RECOMMENDATIONS  AND  CONCLUSIONS 


Test  Workshop 

We  were  fortunate  in  having  many  distinguished 
people  from  both  Government  and  Industry  in  our 
test  workshop.  There  were  four  PHD's  in  the  group 
and,  from  some  of  the  ideas  surfaced,  I suspect 
there  were  a few  more  lurking  in  the  background 
unannounced.  We  began  our  workshop  with  a series 
of  short  presentations  to  orient,  motivate  and  pro- 
vide background  information  to  the  participants. 

Col  Baker  gave  a short  talk  on  the  status  of  MIL- 
STD-1540  (MIL-STD-1540A,  Test  Requirements  for 
Space  Vehicles).  We  talked  a little  about  the 
practical  implementations  of  MIL-STD-1540A  on  our 
GPS  Satellite  Program.  Mr.  Nick  Babich  from  Boeing 
gave  us  an  overview  of  Interface  Testing  experience. 
Mr.  Dean  Lindstrom  of  Hughes  gave  an  interesting 
review  of  the  impact  of  development  testing  on  the 
component  and  system  level  test  program.  Finally, 
Mr.  B.  J.  O'Brien  of  TRW  Systems  gave  us  some  good 
insight  on  the  practical  matters  of  improving  test 
effectiveness.  These  speakers  really  succeeded  in 
getting  the  attention  and  arousing  the  interest  of 
the  group.  Some  very  good  dialogue  developed  in 
the  process.  There  were  a few  more  participants 
than  expected;  however,  our  workshop  was  well 
balanced  among  Air  Force,  NASA,  Aerospace  Corpora- 
tion and  Industry  representatives;  fifty-one  in  all. 
We  enjoyed  excellent  dialogue  with  surprising 
empathy,  understanding  and  appreciation  for  prob- 
lems considered  and  improvements  suggested.  A sig- 
nificant part  of  our  effort  centered  about  MIL-STD- 
1540A;  therefore,  at  the  onset  I would  like  to 
express  a personal  opinion.  Having  implemented 
MIl-STD-1 540A  on  the  GPS  Program,  I think  it  is 
good  document.  It  needs  improvement,  but  on  bal- 
ance, if  its  implementation  is  approached  in  a 
reasonable  and  rational  manner,  it  is  not  nearly  as 
scarey  as  some  people  think  it  is. 

Discussion  Areas 

We  broke  down  into  seven  groups  or  discussion  areas 
as  listed.  Oria’r.ally  we  had  eight  subjects  pre- 
selected. We  ended  up  combining  several  of  the  old 
ones  with  a couple  of  new  ones.  One  new  topic 
involved  "Flying  Qualification  Hardware,"  while 
another  treated  the  impact  of  the  Space  Shuttle  on 
test  programming.  Coverage  of  these  subjects  was 
greatly  enhanced  by  NASA  participants;  our  thanks 
for  their  help,  support  and  contribution. 

Development  Testing 

In  the  area  of  Development  Testing  there  was  a con- 
census that  greater  attention  to  Development  Test 
Planning  is  needed.  Moreover,  improved  planning 
must  be  accompanied  by  greater  discipline  in  the 
accumulation,  application  and  implementation  of 
development  test  data.  One  recommendation  sug- 
gested an  independent  review  of  test  planning  to 
assure  balance  between  the  design  and  practical  test 
viewpoint.  Development  testing  tends  to  be  domin- 
ated by  the  design  operation.  Another  recommenda- 
tion suggested  use  of  development  testing  to  learn 


the  weakness  of  the  design  early  to  either  correct 
or  control.  Postulation  of  non-standard  events  or 
conditions  may  be  very  beneficial  in  learning  the 
weak  link  early  enough  to  characterize  and  control 
or  remove  it.  Finally,  better  salesmanship  and 
cost  analysis  in  the  test  area  was  suggested  in 
order  to  determine  where  the  best  test  investment 
is.  The  general  feeling  expressed  by  the  group 
pointed  to  a greater  return  on  test  investment  "Up 
Front"  where  characteristics  and  capabilities  are 
determined  and  accommodated  as  opposed  to  the 
matter  of  follow-on  individual  compliance  demonstra- 
tion testing. 

Test  Planning 

The  Test  Planning  Group  considered  the  areas  of 
schedule  and  test  changes.  There  was  a feeling 
among  Government  members  that  the  Government  should 
be  open  to  contingency  planning.  This  is  another 
way  of  saying  we  need  more  schedule  realism.  In  a 
competitive  environment  everyone  is  optimistic  and 
schedules  are  generally  success  oriented.  We  like 
to  think  and  project  in  terms  of  success;  however, 
experience  teaches  otherwise.  Excessive  test 
optimism  can  be  a disservice  to  everyone  and  in  the 
long  run  produces  events  compression  and  test 
exigencies  which  tends  to  degrade  mission  assur- 
ance. Test  scheduling  and  planning  should  be 
reconciled  with  experience.  If  you  see  no  failure, 
be  suspicious,  be  aware  and  be  perceptive;  go  back 
and  examine  that  test.  The  other  area  of  consider- 
ation centered  about  test  changes.  Beware  of 
readily  accepting  a test  solution  for  a prior  fail- 
ure problem.  Each  failure  has  its  own  character 
and  conditions.  Be  sure  you  have  considered  and 
identified  the  source  of  the  problem,  confirmed 
the  test  solution  and  have  positive  reason  for  test 
change. 

Test  Standards 

The  area  of  test  standards  brought  out  some  inter- 
esting views  on  the  analysis  of  the  sequence  and 
ordering  of  qualification  and  acceptance  tests. 

The  present  practice  of  acceptance  level  testing  to 
establish  performance  base  for  qualification  test, 
followed  by  acceptance  test  to  demonstrate  survival 
and  performance  was  judged  to  be  an  inefficient  use 
of  test  facilities.  The  possibility  of  establish- 
ing the  initial  operational  test  status  of  the  test 
item  under  qualification  levels  would  effectively 
combine  tests  and  in  many  instances  improve  the 
efficiency  and  economy  of  test  with  no  impact  on 
mission  assurance.  The  recommendation  to  permit 
this  option  was  advanced  to  enhance  the  effective- 
ness and  economy  of  test  programming  and  utilization 
of  test  facilities.  Another  consideration  Involved 
system  level  thermal -vacuum  testing.  Testing  under 
vacuum  can  be  very  expensive  when  running  thermal 
cycling.  It  was  recommended  that  an  option  be  pro- 
vided for  running  the  thermal  cycling  at  ambient 
pressure  providing  you  have  run  both  hot  and  cold 
soak  tests  under  vacuum.  Additionally,  it  was  sug- 
gested this  approach  might  be  more  perceptive 
because  of  the  added  confidence  in  having  achieved 
and  stabilized  at  the  temperature  extremes. 

The  subject  of  Burn-in  testing  was  hotly  debated. 

The  current  requirement  of  MIL-STD-1 540  is  300  hours. 
The  subject  continues  to  be  controversial  in  a gross 
sense  primarily  because  of  insufficient  data  to 
establish  where  we  are  in  terms  of  optimum  Burn-in 
time  for  the  items  under  test.  This  subject  was 
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recommended  for  further  study.  Another  area 
identified  for  further  study  is  the  sequence  of 
shock  and  vibration  testing.  It  was  suggested  that 
the  sequence  of  vibration  followed  by  shock,  which 
is  the  sequence  usually  encountered  during  launch, 
might  not  be  the  most  perceptive  in  discovering 
problems.  It  was  suggested  that  shock  followed  by 
vibration  would  be  a more  meaningful  test  sequence 
in  terms  of  mission  assurance.  It  was  recommended 
this  area  be  examined  to  determine  if  sequence 
change  will  enhance  mission  assurance. 

Fly  What  You  Qualify 

NASA  Goddard  has  seven  years  experience  with  what 
they  call  the  "Protoflight''  concept.  Simply  stated 
it  means  "Fly  What  You  Qualify."  It  is  an  inter- 
esting concept  worthy  of  further  evaluation.  Flight 
history  data  and  test  to  maximum  predicted  flight 
environment  plus  a 3 DB  margin  proved  successful. 
There  is  risk  of  overstressing  components  and  it  is 
prudent  to  approach  the  possibilities  cautiously. 

Be  careful  about  levels.  Present  Air  Force  policy 
for  Space  Flight  is  to  completely  refurbish  quali- 
fication test  items  prior  to  space  flight  readiness 
certification.  The  Fly  What  You  Qualify  concept 
offers  some  interesting  possibilities  for  simplifi- 
cation in  the  process  of  flight  certification  of 
qualification  hardware.  Further  discussion  was 
centered  about  the  Space  Shuttle  impact  on  testing. 
We  are  lacking  specific  data  on  environments  and 
shuttle  peculiarities.  However,  we  are  provided  an 
opportunity  for  the  first  time  to  conduct  on-orbit, 
pre-deployment  test  and  checkout.  We  must  use  this 
added  advantage  wisely  and  avoid  undue  relaxation 
or  elimination  of  ground  testing  in  light  of  on- 
orbit  capabilities  and  recovery  option.  A low 
earth  orbit  launch  complex  offers  some  interesting 
challenges  to  effective  and  economical  testing  of 
space  vehicles. 

Test  Problems  and  Equitable  Retest 

This  discussion  related  primarily  to  MIL-STD-1540 
and  the  sometimes  unpleasant  but  very  real  consider- 
ations relating  to  test  problems  and  equitable 
retest.  There  is  need  to  strengthen  the  retest 
criteria  presently  included  in  MIL-STD-1540.  At 
present  almost  every  retest  is  a negotiated  solu- 
tion. The  type  and  extent  of  retesting,  particu- 
larly following  rework,  needs  further  consideration 
and  the  development  of  a discipline  pattern  within 
MIL-STD-1540. 

Discussion  then  shifted  to  the  issue  of  test  plan- 
ning and  its  role  in  the  design  review  process. 
Testing  is  not  something  that  occurs  automatically. 
It  must  be  addressed  with  the  same  rigor  as  the 
basic  design  and  must  reflect  a consistent  and  com- 
patible Interface  with  the  design.  It  must  be 
planned  and  disciplined  and  still  remain  suffi- 
ciently flexible  to  accommodate  historical  variables 
without  sacrifice  to  the  effective  and  economical 
achievement  of  all  test  objectives.  This  mandates 
early  and  intensive  test  planning  and  review  effort. 
It  was  suggested  that  an  independent  evaluation  of 
test  planning  in  the  early  stages  of  the  design 
process  would  add  value  to  the  test  planning  and 
test  programming  efforts  and  products. 

The  next  topic  discussed  involved  the  matter  of 
Stored  Space  Vehicles.  Time  s to  create  fail- 
ures in  storage  notwithstanding  ,.  • most  diligent 
preparation  and  implementation  o storage 


discipline.  The  possibility  of  time/temperature 
dependent  mechanisms  working  and  other  subtle 
aspects  of  time  storage  impact,  prompted  the  feel- 
ing among  participants  that  source  re-test  is 
necessary  after  six  months  of  storage.  While  there 
were  some  special  factors  to  be  considered,  it  was 
generally  agreed  that  a functional  verification 
test  under  ambient  conditions  would  suffice,  pro- 
viding that  critical  environmentally  activated 
conditions/events  were  properly  checked.  The  type 
and  extent  of  Re-Test  and  the  extent  of  environ- 
mental conditions  required  is  dependent  on  analysis 
and  characterization  of  the  item  under  considera- 
tion. MIL-STD-1540  should  provide  for  Re-Test 
considerations  appropriate  for  stored  vehicles. 

Interface  Testing 

The  matter  of  interface  testing  and  specific 
requirements  relating  to  the  interface  tend  to  be 
neglected.  There  is  need  to  consider  functional 
interactions  and  reactive  variables  as  well  as  fit 
and  form  prior  to  the  point  where  correction 
becomes  an  expensive  and  time  consuming  proposi- 
tion. There  is  need  for  additional  treatment  of 
Interface  Test  Criteria  and  requirement  in  MIL-STD- 
1540  to  assure  early  consideration  of  functional  as 
well  as  physical  compatibility  across  interfaces. 

Test  Data  generation  and  evaluation  was  then  con- 
sidered by  the  group.  It  was  a general  consensus 
that  too  much  reliance  is  placed  on  gross  test 
results  without  sufficient  attention  to  variations 
within  the  tolerance  bands.  Analysis  to  determine 
significance  of  signs  relating  to  test  Instability 
(trends  within  acceptable  tolerance  limits)  can  be 
of  value  in  characterization  and  Identification  of 
incipient  failures.  Often  we  complete  a test  suc- 
cessfully with  only  a cursory  review  of  data.  We 
tend  to  ignore  data  from  successful  tests,  we  con- 
centrate on  failure.  The  group  recommends  review 
of  critical  parameter  test  data  for  anomalous  indi- 
cations and  analysis  to  determine  probable  cause 
and  significance  even  though  the  test  was  successful 
Look  at  Successful  Test  Data.  It  was  suggested  that 
the  establ ishment  of  minimum  standards  for  data 
review  should  be  developed  and  applied. 

In  closing,  it  has  been  said  that  if  a single  idea 
comes  out  of  this  conference,  it  has  been  well 
worthwhile.  I agree  with  that  statement;  however, 
even  if  no  new  ideas  were  developed,  the  free 
exchange  of  ideas  and  dialogue  between  government 
and  industry  still  make  it  all  worthwhile. 


DISCUSSION  AREAS 

• Development  Testing 

• Test  Planning 

• Test  Standards 

• Flying  Qualification  Hardware 

• Shuttle 

• Test  Problems  & Equitable  Retest 

• Interface  Testing 
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DEVELOPMENT  TESTING 

• Greater  attention  to  dev.  test  planning 
needed 

• Govt  could  lead  in  assuring  adequate 
schedule  & discipl ine 

• Need  an  independent  review  of  test 
planning  (test  viewpoint  vs.  design) 

• Off-Limits  testing  at  dev.  may  be  very 
beneficial 

Postulate  some  non-std  events  - e.g. 
Power  Interrupt,  Very  low  voltage 
drop,  transient,  etc.  Learn  the 
weak  link  early. 

• Better  cost  analysis  of  testing  would 
help  show  where  the  best  test  investment 
is.  We  suspect  more  up-front  is  more 
cost  effective. 


TEST  PLANNING 


• Schedule 

Govt,  mgmt  should  be  open  to 
contingency  scheduling  for  failures 
A retest.  AF  INPUT! 

Need  more  schedule  realism  - beware 
excess  optimism  when  justifying 
schedule,  reconcile  with  failure 
experience  if  there  were  no  failures. 
Re-examine  test. 

• Test  Change 

Beware  readily  accepting  a test 
solution  for  a prior  failure  problem. 

Make  sure  there  is  a reason  to 
change. 


TEST  STANDARDS  RECOMMENDED  CHANGES 
TO  MIL  STD  1540 


• Environmental  acceptance  & qualification 

testing  (components) 

• Permit  option  of  accept  A qual,  each 
env.  in  turn,  as  alternative  to  com- 
plete ATP  followed  by  complete  qual. 
program 

• Rationale  Practicality  . . . 

• System  Thermal  Vacuum  Acceptance  Testing 

• Permit  option  of  thermal  cycling  at 
ambient  provided  hot  A cold  soaks 
under  TV  also  conducted 

• Rationale  - Problem  in  reaching  accept 
temps  on  components  in  many  space  veh. 
designs . 


RECOMMENDED  FOR  FURTHER  STUDY 


• Burn-In-Testing 

Industry  experience  on  burn-in  failures 
be  researched  to  establish  a realistic 
and  sufficient  duration  for  burn-in 

(now  300  hours) 

• Sequence  of  Testing  Shock  A Vibration 

Shock  Then  vib  Perceptive  sequence 

Vib  then  shock  usual  system  sequence 


FLY  WHAT  YOU  QUALIFY 

• NASA  Goddard  has  7 years  experience 
w/"Protofl ight"  concept 

e Flight  history  equiv.  to  separate  qual. 
hdw. 

• Risk  of  overstressing  components  be 
careful  about  levels. 

• SAMSO  should  consider  this  concept 


SHUTTLE  IMPACT 

• We  lack  data  on  environments 

• Beware  excess  relaxation  of  grd.  Test 
due  to  on-orbit  pre-deployment  check 
abi  1 i ty. 


TEST  PROBLEMS  A EQUITABLE  RETEST 

Retest  guidelines  in  1540  could  be 
strengthened  for  component  retest 

For  space  vehicle  testing  every  retest 
is  really  a negotiated  solution 

Multiple  retests  following  rework  need 
exploration  and  possible  treatment  in  1540 

Test  planning  should  be  addressed  at  PDR 
to  the  same  rigor  as  the  design 

Stored  Space  Vehicles 

• Time  seems  to  create  some  failure 
mechanisms 

• Storage  6 mos.  requires  some  retest 

• Non- environmental  functional  verifi- 
cation is  probably  adequate 

• Extent  of  test  & possible  environ- 
ments depends  on  complexity  of  space 
vehicle 

• Consideration  for  test  of  stored 
vehicles  should  be  incorporated  into 
MIL-STD-1540 


t 
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INTERFACE  TESTING 


• Interface  testing  (specific  requirements) 
tend  to  be  neglected 

• Early  treatment  needed 

Consider  functional  interactions, 
not  just  physical 

• Some  treatment  in  MIL-STD-1540  is 


desired 


TEST  DATA 


• Data  from  "successful  tests"  often 
ignored 

• Examine  "Good"  data  for  trends  within 
the  acceptable  boundaries 

• Some  minimum  STANDARDS  for  data  review 
would  help 
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Abstract 

A short  description  and  tabular  information 
are  provided  for  five  Lockheed  employee  centered 
motivation  programs  which  contribute  to  increased 
productivity,  cost  savings,  and  hardware  quality. 
Eight  lessons  learned  are  reviewed  to  aid  other 
organizations  in  developing  programs  which  will  be 
the  most  effective  in  assuring  mission  success  and 
in  increasing  the  quality  of  work.  It  is  concluded 
that  such  motivational  programs  can  increase  pro- 
ductivity, improve  employee  attitudes,  reduce 
defects  and  accidents,  and  assure  high  probability 
of  mission  success.  However,  these  activities  re- 
quire strong  management  commitment  including  budget, 
time,  support  personnel  and  active  follow-up. 

Introduction 

Industrial  motivation  and  training  are  tradi- 
tionally thought  of  in  terms  of  awards,  management 
goals,  financial  incentives  and  formal  courses. 
However,  it  can  easily  be  demonstrated  that  this  is 
a systems  problem,  not  unlike  complex  hardware, 
involving  technology  and  structure  as  well  as  peo- 
ple. Most  organizations  have  evolved  from  product 
or  functional  requirements,  from  the  attempt  to 
impose  traditional  pyramid  structures,  or  from  the 
idiosyncracies  of  executives.  The  requirements  and 
needs  of  all  the  people  in  the  organization  are 
rarely  considered  in  such  organizational  decisions. 
Back  in  1763,  Adam  Smith  in  his  book,  The  Wealth  of 
Nations,  emphasized  the  point  that  ?aopl9  are  moti- 
vated by  self  interest.  Unfortunately  we  often 
forget  this  fact  in  the  evolution  of  our  organiza- 
tions and  the  structuring  of  our  motivation  programs. 

Motivation  Pragrumn 

At  Lockheed  Missiles  and  Space  Co.,  (LMSC)  in 
Sunnyvale,  California  we  have  recently  attempted  to 
utilize  the  principles  of  greater  employee  partici- 
pation and  involvement,  reduced  arbitrary  controls 
and  increased  organizational  linking  in  a number 
of  programs  including: 


A employee  participation  program  started  in 
Japan  involving  the  training  of  workers  in  quality 
engineering  techniques  and  then  applying  these 
techniques  in  weekly  meetings  on  a voluntary,  but 
paid  basis,  to  analyze  and  solve  quality  and  pro- 
duction problems. 


Employees  meeting  with  their  supervisor  and 
a facilitator  for  one  hour  a week  for  8-12  weeks 
to  discuaa  work  Impediments  and  explore  potential 
solutions.  Multi-level  meetings  are  held  monthly 
to  bring  the  most  difficult  problems  to  the  atten- 
tion of  other  organizations  and  higher  management. 


Upward  Communications 

Skip-level  meetings  between  10-16  randomly 
selected  salaried  employees  and  their  respective 
Vice-President.  Each  employee  brings  one  positive 
and  one  negative  aspect  of  the  job  to  the  meeting. 
These  are  posted,  discussed  and  actions  designated. 

Upward  Performance  Evaluation 

A voluntary  program  whereby  a manager  or 
supervisor  can  be  evaluated  by  hi3  subordinates  on 
fifteen  variables.  He  is  privately  counseled  on  the 
results  which  include  the  profiles  of  his  own  eval- 
uation vs.  the  mean  values  of  his  subordinates' 
ratings. 

Potential  Damage  Evaluation 

Employees  at  all  levels  who  are  familiar  with 
an  operation,  evaluate  the  potential  for  damage  to 
hardware  or  injury  to  personnel  on  a three  point 
scale  for  each  subfunction  in  the  process.  This 
sensitizes  the  employees  to  safety  and  provides 
management  with  a prioritized  list  for  eliminating 
potential  hazards. 

All  of  these  programs  are  directly  related  to 
creating  and  maintaining  products  of  the  highest 
level  of  quality  and  thus  are  of  interest  to  atten- 
dees of  a Mission  Assurance  Conference.  Figure  1 
presents  a summary  of  important  parameters  related 
to  these  programs.  Mare  details  on  the  mechanics, 
training  and  outcomes  of  these  efforts  may  be  ob- 
tained from  the  articles  listed  in  the  first  four 
references  at  the  end  of  the  paper  or  from  direct 
inquiries  to  the  author. 

Lessons  Learned 

It  is  felt  that  the  remainder  of  this  paper 
can  most  beneficially  be  devoted  to  lessons  learned 
over  the  past  ten  years  since  a company  or  govern- 
ment jurisdiction  would  probably  wish  to  tailor  an 
internal  motivation  program  to  their  own  require- 
ments and  environment. 

Nothing  Lasts  Forever 

All  motivation  and  training  programs  such  as 
Zero  Defects,  Extra  Care,  or  Q.C.  Circles  become 
somewhat  'old  hat'  and  tend  to  lose  their  impact 
after  3-4  years.  The  less  structured  the  program 
the  greater  the  potential  for  a longer  life.  How- 
ever, the  best  technique  is  to  either  put  a time 
limit  on  the  program  at  the  outset  or  modify  the 
format  and  schedule  after  an  extended  period. 

Management  Values  and  Involvement 

There  must  be  conplete  program  acceptance  and 
active  participation  by  all  members  of  management 
from  the  Chairman  of  the  Board  down  to  first  level 
supervisors.  If  such  programs  receive  only  per- 
functory lip  service  or  tacit  approval  from  company 
executives,  they  are  doomed  to  failure  from  the  out- 
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Figure  1 Recent  Lockheed  Motivation  Programs 
Period  Participants  Frequency  Measures  of  Effectiveness 


Q.C.  Circles 

1973  - 

Present 

310  Manufac- 
turers on 

Trident  Missile 

Weekly 

Meetings 

o Cost  Benefit  Ratio  of  6:1 
o 90 ? of  Participants  Feel 
Program  Effective 
o Reduction  in  Defects  per 
Man  Hour  Worked 

TEAM 

1969  - 
1974 

3200  Hourly 
& Salaried 

Mfg,  PA,  Eng, 
Programs, 
Procurement,  etc. 

Weekly 

Meeting 

8-12  Weeks 

o 77?  Problems  Resolved 
o Attitude  Surveys  - TEAM 
Groups  Signif.  Higher  than 
Controls 

o Multimillion  Dollars 
Savings 

Upward 

Communications 

1972  - 
Present 

785  Salaried 
Employees  - All 
Organizations 

1 per 

Month 

o 60?  Problems  Resolved 
o 89?  of  Participants  Rated 
Program  as  Beneficial 

Upward 

Performance 

Evaluation 

1973  - 
1977 

850  Bnployee 

Raters, 

70  Mgr/Supv, 

Ratees 

10-20  per 
Year 

o Subjective  Evaluations 
from  Quployees  and 

Managers  - Very  Positive 

Potential 

Damage 

Evaluations 

1970  - 

Present 

Approx  30 

Employees  per 

Study  plus 

1 Full  Time 

Human  Engineer 

Approx  1 
per  Year 

o Reduced  Rework,  Rejects 
& Accidents 
o Improved  Employee 

Attitudes 

o Better  Methods  of  Fabri- 
cation & Assembly 
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set.  Our  most  successful  efforts  at  LMSC  have 
occurred  uhen  Directors  and  Vice  Presidents  have 
attended  meetings  and  employee  presentations,  or 
have  established  their  own  groups  from  meml-ers  of 
their  immediate  staff.  It  is  also  important  to 
bring  all  power  structures/decision  makers  together 
in  support  of  a motivation  program.  For  example, 
an  advisory  group  representing  the  customer,  company 
and  union  should  be  established.  If  we  could  point 
to  one  mistake  which  has  been  most  detrimental  in 
our  TEAM,  Upward  Communications,  and  QC  Circle 
efforts  it  would  be  the  structuring  of  programs 
which  didn't  actively  involve  middle  management  on 
a regular  basis.  This  produces  communication  chan- 
nels between  first  line  personnel  and  executive 
level  management  with  mid-management  feeling  left 
out  of  lacking  control  over  information  transfer. 
Unless  people  in  an  organization  feel  extremely 
secure,  this  practice  can  produce  foar  and  covert 
retaliation  by  'old  guard'  line  managers. 

The  Shoemaker  Knows  Shoes  Best 

We  have  been  consistently  amazed  and  impressed 
by  the  high  calibre  and  technical  depth  of  processes, 
techniques,  and  new  innovations  which  have  been 
formulated  by  working  fabricators,  assemblers, 
technicians,  and  engineers  in  group  meetings  and  in 
individual  interviews.  One  of  the  most  frequent 
concerns  exhibited  in  Upward  Communications  meet- 
ings involves  management  or  the  customer  not  re- 
questing technical  information  directly  from  the 
expert  when  a problem  exists  on  a spacecraft  or 
when  a presentation  is  made.  When  will  the  Air 
Force  Captain  and  industry  Senior  Engineer  be  per- 
mitted to  solve  the  problem  on  a pin-puller  design 
at  their  level  rather  than  trying  to  resolve  this 
technical  problem  at  the  General/Vice  President 
level? 

Real  vs  Stated  Costs 

This  is  a double  edged  problem.  On  one  hand 
we  find  that  stated  costs  frequently  don't  include 
the  time  of  supervision  and  other  indirect  person- 
nel, training  materials,  employees  time  in  meetings 
and  resolving  problems,  or  program  evaluation  exer- 
cises. On  the  other  hand,  workers  who  are  'turned 
on'  by  their  jobs  and  the  opportunity  to  exhibit 
some  creativity  or  decision  making  may  be  ^0-^0f 
more  productive.  One  company  found  that,  when  they 
reduced  the  number  of  hours  from  40  to  36  (with  a 
commensurate  reduction  in  pay)  rather  than  laying 
people  off  during  a downturn  in  sales,  the  employ- 
ees were  more  dedicated  and  actually  produced  at  a 
higher  rate  than  before.  In  summary,  true  system 
costs  should  be  computed  on  a total  input  vs  total 
output  basis  rather  than  on  partial  program  costs 
or  a single  cost  improvement. 

Evaluation  - Necessary  But  Simple 

It  is  highly  desirable  to  continually  audit 
and  evaluate  motivation  programs.  In  fact,  the 
initial  evaluation  procedure  should  be  developed 
concurrently  and  as  an  integral  part  of  the  Pro- 
gram Plan.  Management  will  demand  some  criteria 
of  effectiveness  to  justify  the  program  costs. 
However,  cost  saving  estimates,  grievances  before 
and  after,  work-rework  times,  discrepancy  rates, 
attitude  survey,  etc  should  be  kept  simple.  Our 
experience  has  shown  that  most  Quality  Assurance 
or  Finance  data  collection  methods  are  not  appli- 
cable for  this  purpose  and  are  far  too  detailed. 

Also,  the  formal  Cost  Improvement  Program  requires 


too  much  paper  work  and  administrative  approval 
levels  to  be  worth-while.  On  one  program  we  found 
that  7*>f  of  the  support  time  was  spent  in  devel- 
oping cost  improvement  .lata.  A good  rule  of  thumb 
is  to  not  exceed  104  of  the  total  program  cost  or 
time  on  evaluation. 

Programs  Don't  Hut.  Tenge' vf" 

The  most  common  misconception  is  that  once  a 
motivation  program  is  planned  and  started  in  the 
line  organization,  it  will  perpetuate  itself  on  a 
continuous  tasis  - like  a I reader  reactor.  Unfor- 
tunately, at  least  during  the  first  few  years  of  a 
program,  facilitators  are  needed.  On  TEAM  and  Q.C. 
Circles  we  found  that  a good  ratio  was  1 facili- 
tator for  every  4-6  groups.  Their  functions  are 
to  schedule  meetings,  obtain  equipment  and  faci- 
lities, help  keep  meetings  on  the  objectives, 
counsel  with  the  leader  and  group  members,  obtain 
outside  experts,  coordinate  with  other  groups,  and 
aid  in  follow-up  activities.  These  facilitators 
have  to  understand  the  technology,  the  company 
organization  and  dynamics,  and  human  behavior. 

Secondary  Benefits 

Although  we  have  directed  the  discussion  of 
motivation  programs  toward  the  improvement  of 
Mission  Assurance  through  increased  human  and 
hardware  reliability,  there  are  a number  of  im- 
portant secondary  benefits  from  such  efforts 
including: 

Supervisory  Development.  On-the-j ob- training 
in  listening  and  dealing  with  employees  and  ob- 
taining counseling  and  feedback  from  the  facili- 
tator. 

Productivity.  The  removal  of  impediments 
to  getting  the  job  done  and  giving  people  a larger 
'piece  of  the  action'  are  fundamental  factors  in 
increasing  productivity. 

Opening  Up  Communications.  Once  people  see 
that  management  is  receptive  to  changes  and  new 
ideas,  vertical  and  horizontal  communications  will 
improve  dramatically. 

Integrate  Management  and  Employee  Goals. 

In  analyzing  and  researching  possible  solutions  to 
problems,  employees  gain  a better  insight  into  the 
constraints  and  forces  with  which  management  must 
contend.  Experience  showed,  that  after  investi- 
gating working  procedures  and  methods,  TEAM  groups 
would  recommend  that  about  255f  of  these  systems  be 
left  intact  because  they  represented  the  most  ef- 
fective alternative.  Regarding  increased  manage- 
ment insight  to  employee  goals,  in  an  Upward 
Communications  meeting  one  of  our  Vice  Presidents 
was  asked,  "Have  you  heard  anything  in  this  meet- 
ing which  your  were  completely  unaware  of?" 

After  thinking  about  it  for  a minute  he  responded, 
"No,  but  I certainly  got  a different  slant  on  some 
of  these  problems  when  I heard  them  described  by 
you  people  on  a face-to-face  basis." 

Must  Become  Lifestyle 

In  the  view  of  the  author,  when  considering  long 
term  future  organizational  trends  (10-20  years 
hence)  motivation  programs  as  described  in  this 
paper  may  be  viewed  in  the  future  as  merely  crut- 
ches or  interim  measures  In  the  evolution  of  more 
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productive  and  socially  responsive  organizations. 
Perhaps,  the  day  has  already  arrived  for  elimi- 
nating or  modifying  such  rigid  structured  programs 
as  Zero  Defects,  Extra  Care,  Cost  Improvement 
Quotas,  etc.  Companies  such  as  Donnelly  Mirrors, 
Hewlett  Packard  and  Tektronix,  who  have  evolved 
company  wide  profit  sharing  and  participative 
management  practices  over  the  years,  are  probably 
not  good  candidates  for  such  programs  as  TEAM  or 
Q.C.  Circles  since  they  have  already  achieved  a 
type  of  corporate  self-actualization. 


5.  D.  A.  Stuart,  "M5D  Management  or  What  Have  We 
Here?"  LMSC  Report  (*©657727,  August  1977 
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Unfortunately,  most  of  the  large  aerospace 
companies  historically  were  conceived  during  the 
period  when  autocratic  methods  were  taken  for 
granted.  Also,  their  customer  has  traditionally 
been  the  Department  of  Defense  or  other  like 
government  agencies;  and  therefore,  they  have 
tended  to  be  structured  and  to  operate  in  a para- 
military style.  We  see  significant  changes  occur- 
ring at  Lockheed  and  other  similar  companies.  An 
example  of  this  is  an  internally  distributed  book 
which  was  written  by  Dr.  Deraid  Stuart,  Vice  Pre- 
sident and  General  Manager  of  LMSC's  Missile 
Systems  Division.  In  the  book  (Ref.  5)  Dr.  Stuart 
writes,  "I  am  a strong  supporter  of  the  concept  of 
management  by  negotiated  objectives  - and  I firmly 
believe  that  the  methods  of  accomplishment  of 
objectives  are  critical.  The  end  results  do  not 
always  justify  the  means  used.  Hence,  I try  to 
generate  and  support  an  environment  and  moral  cli- 
mate that  allows  all  to  be  proud  to  be  part  of  the 
organization.  I believe  that  one  of  management's 
most  important  tasks  is  to  protect  the  future  and 
that  can  best  be  done  by  developing  and  depending 
upon  strong  (competent)  people.  Hence,  I try  to 
generate  and  encourage  the  personal  development 
and  growth  of  all  members  of  the  organization.  I 
believe  I can  summarize  the  above  by  saying  that 
I foster  building  dedicated  (product  and  program 
oriented)  teams  and  finding  win-win  solutions.  I 
continually  push  for  and  encourage  activities, 
policies  and  actions  that  allow  everyone  to  be 
proud  of  what  he's  doing  - thus  allow  them  to  have 
'fun'  rather  than  just  'do  a job.'  Participative 
management,  management  by  objectives,  management 
(team)  development,  quality  circles,  etc.,  are  all 
ways  of  letting  the  doers  'have  fun'  and  'do  their 
own  thing'  to  the  organization's  benefit  - instead 
of  them  'Just  working'."  Prom  personal  experience 
the  author  can  state  that  Deraid  'practices  what 
he  preaches.'  With  executives  and  managers  of 
this  stature  we  can  hopefully  look  forward  to  both 
a higher  quality  of  work  and  a higher  quality  of 
life  in  the  future. 
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Quality  Circles  at  Hughes  Aircraft 
William  E.  Courtright 
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Abstract 

The  success  of  quality  circles  at  Hughes 
Aircraft  Company  supports  the  goals  of  this  SAMSO 
MISSION  ASSURANCE  PROGRAM.  For  example,  the  Hughes 
Quality  Circles  approach  helps  to  overcome  the 
present  counter  productive  communications  and 
motivational  practices  and  it  establishes  a new 
cost  effective  method  that  helps  solve  problems 
and  assure  mission  success. 

Introduction 

In  early  1976,  the  Hughes  Aircraft  Company 
became  aware  of  the  progress  that  the  Lockheed 
Missile  and  Space  Company  was  making  with  their 
Quality  Circle  Program.  A trip  was  made  to 
Lockheed's  Missile  Systems  Division  at  Sunnyvale, 
California  in  order  to  get  a firsthand  look  at 
their  QC  Circle  Motivation  Program.  Shortly 
thereafter  Hughes  researched  all  available  liter- 
ature on  participative  motivation  programs  including 
a detailed  review  of  the  Japanese  QC  Circle  material 
under  J.U.S.E.,  Joint  Union  of  Science  and  Engineers. 
With  this  combined  information  on  motivation  pro- 
grams, Hughes  developed  a Quality  Circle  Program 
and  started  a pilot  run  in  September  1976  at  the 
Hughes  Carlsbad,  California  plant,  the  location 
of  the  Industrial  Products  Division. 

Implementation  of  the  Quality  Circle 
Program  at  Hughes 

Pilot  Program  at  Carlsbad 

The  pilot  Quality  Circle  approach  included 
briefing  the  general  manager  as  well  as  other  top 
management  at  Carlsbad  on  all  aspects  of  the  pro- 
gram and  receiving  their  approval  for  proceeding 
with  the  program.  All  other  concerned  management 
and  employees  were  trained  at  the  end  of  1976  and 
four  Quality  Circles  were  started  in  January  1977. 


The  Ground  Systems  Group's  Division  15  Manufacturing 
at  Fullerton,  California  and  the  Tucson  Manufactur- 
ing Division  at  Tucson,  Arizona,  have  quality 
circles  underway  and  other  Hughes  Groups  and  Di- 
visions such  as  Space  and  Communications  are  investi- 
gating the  program  and  have  shown  interest  in  be- 
coming involved.  At  this  time  over  eleven  Hughes 
divisions  with  about  250  people  are  taking  part  in 
the  Quality  Circle  program  in  the  manufacturing 
areas.  These  Hughes  Divisions  are  involved  with 
all  aspects  of  products  and  research  of  electronic 
and  mechanical  applications  for  commercial  and 
military  purposes. 

Results 

At  the  outset  of  the  quality  circle  program 
there  was  some  concern  by  management  for  the  name 
of  the  program.  Some  felt  with  good  reason  that 
maybe  the  circle  should  be  called  a communication 
circle.  Hughes  experience,  however,  after  two 
years  has  been  that  Quality  Circle  or  QC  Circle  as 
a name  is  very  acceptable.  The  word  quality  is 
helpful  because  employees  and  union  people  respect 
the  importance  of  quality  and  they  know  that  the 
circle  program  is  not  aimed  at  speeding  up  work. 
Although  product  quality  is  of  primary  concern  to 
the  circle,  the  circle's  effort  is  also  involved 
with  any  problem  that  affects  the  performance  of 
the  individuals  or  the  group.  Some  results  of  the 
quality  circle  program  are: 

' Pre-amp  spacers  cracked  and  breaking 
during  swaging  operation  requiring 
excessive  rework  - changed  material. 

Excessive  amount  of  defective  material 
supplied  to  assembly  line  - supplier 
took  corrective  action. 

Adequate  tools  were  needed  for  assembly  - 
necessary  tools  were  provided. 


Expansion  into  Industrial  Electronics  Group 
and  Other  Hughes  Groups  and  Divisions 

The  Carlsbad  Quality  Circle  program  was  so 
successful  as  a pilot  program  that  it  was  expanded 
in  June  1977  to  four  other  divisions  in  the  In- 
dustrial Electronics  Group.  These  four  other 
divisions  included  the  Connecting  Devices  Division 
at  Irvine,  California;  the  Solid  State  Products 
Division  at  Irvine,  California;  the  Microelectronic 
Products  Division  at  Newport  Beach,  California; 
and  the  Electron  Dynamics  Division  at  Torrance, 
California.  Since  June  1977,  Division  79's  Electro- 
Optical  6 Laser  Production  at  El  Segundo,  Calif- 
ornia; Division  26's  F-14  production  at  El  Segundo, 
California;  the  Santa  Barbara  Research  Center,  at 
Santa  Barbara,  California  and  Spectrolab  at 
Sylmar,  California  have  initiated  quality  circles. 


Chemical  cleaning  procedures  reviewed  on 
volume  priority  basis  - improves  methods, 
yield,  and  quality. 

Front  end  assembly  procedure  simplified 
from  three  Pi's  to  one  PI  - improves 
operator's  understanding  and  performance. 

Sample  boards  redesigned  for  assembly  - 
improved  quality  and  assembly  performance 
large  savings  - $48, 000/year . 

Ultrasonic  cleaner  used  to  clean  parts  - 
major  reduction  of  defects  and  large 
savings  - $31 ,000/year. 
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Developed  step-by-step  manufacturing 
assembly  procedures  with  photographs  - 
improves  training  and  performance. 

Inadequate  space  - improved  through 
brainstorming. 

A classification  system  was  developed 
to  identify  tools  and  equipment  - 
saves  time. 

.Noise  levels  were  reduced  - improves 
assembly  performance. 

Bonding  manufacturing  procedures  not 
followed  and  inadequate  - changes 
improve  quality  and  performance. 

. Packaging  and  planning  problems  were 
identified  - changes  improve  assembly 
quality  and  performance. 

Quality  control  charts  for  major  problem 
circuit  boards  displayed  - improved 
operator  performance  and  quality. 

Provided  resistor  color  coding  guide- 
lines - improved  assembly  quality  and 
yield. 

Parts  list  information  made  available 
for  assembly  personnel  - improved 
performance. 

In  the  quality  circle  program  at  Hughes  we  see 
a significant  increase  in  the  interest  that  circle 
people  show  in  their  jobs.  This  means  that  people 
involved  in  the  program  become  more  committed  to 
doing  their  jobs  properly.  The  circle  program 
significantly  improves  communications  between 
supervisors  and  employees,  leading  to  improved 
operations  and  savings.  Although  some  of  the 
benefits  of  quality  circles  are  not  measurable, 
Hughes  does  use  its  C.I.P,  (Cost  Improvement 
Program)  and  P.I.P.  (Performance  Improvement 
Program)  to  help  measure  its  results. 

Time  spent  by  employees  in  the  quality  circle 
program  is  compared  to  measurable  costs  and  benefits 
from  the  program.  Savings  at  this  time  are  better 
than  4 to  1.  Individual  quality  circles'  vary 
significantly  on  cost  benefits  depending  on  the 
manufacturing  area  involved  and  the  nature  of  the 
circle.  One  circle  had  a savings  of  120  to  1 
another  30  to  1.  Some  circle  savings  are  difficult 
to  measure.  In  all  cases,  however,  the  circles 
contribute  to  improved  quality  and  performance. 

Hughes  Quality  Circle  Approach 

The  Hughes  quality  circle  program  is  similar 
to  Lockheeds'  and  the  Japanese.  Hughes  agrees  that 
training  is  very  important  to  the  effectiveness  of 
quality  circles  and  it  focuses  its  training  towards 
developing  a team  of  people  that  work  well  together 
in  identifying  and  solving  problems.  Hughes 
simplified  training  approach  is  directed  at  pareto 
analysis  as  a management  tool  for  problem  identifi- 
cation and  priority  setting  and  also  it  focuses  on 
cause  and  effect  diagraming  as  a tool  to  help  team 
participation  for  problem  solving.  The  Hughes 
training  approach  utilizes  the  existing  skills  and 


talents  of  the  individuals  in  the  circle  in  order 
to  develop  a team  that  can  freely  and  effectively 
identify  and  solve  problems. 

The  Hughes  Quality  Circle  has  from  S to  10 
people  that  volunteer  for  the  circle  and  who  do 
related  type  work  in  the  manufacturing  area.  The 
manufacturing  circle  meets  with  their  supervisor 
and  a facilitator  three  weeks  of  the  month  for  1/2- 
hour  to  1-hour  leaving  the  last  week  free  for  month- 
ly schedule  deliveries.  Many  circles  start  with  a 
1-hour  meeting  and  adjust  down  to  1/2-hour  as  the 
scope  of  problems  are  reduced.  The  supervisor  is 
leader  of  the  quality  circle  and  in  this  manner 
line  management  responsibility  is  not  changed.  The 
facilitator  has  several  jobs  that  are  important  to 
the  performance  of  the  quality  circle.  First,  the 
facilitator  should  be  trained  in  quality  circles 
and  therefore  he  provides  an  avenue  for  training 
the  circle  in  its  methods  of  defining  and  solving 
problems.  Secondly,  the  facilitator  acts  as  a 
middle  person  between  the  individual  members  such 
as  the  supervisor  and  employee.  In  this  role  the 
facilitator  can  be  very  helpful  in  increasing 
communications  from  very  member  of  the  circle.  In 
this  communication  process  the  facilitator  trains 
everyone  to  realize  the  importance  of  listening  to 
everybody's  ideas.  Finally,  the  facilitator  should 
spend  about  1 hour  for  every  1/2  hour  spent  in 
meetings, in  helping  the  supervisor  and  members  in 
solving  the  circles'  agreed  to  problems  during  the 
week. 

A question  may  be  asked,  "Why  does  the  quality 
circle  program  work?"  It  works  because  the  phi- 
losophy of  the  program  is  that  people  will  take 
more  interest  and  pride  in  their  work  if  they  can 
influence  that  work.  The  quality  circle  program  is 
organized  to  encourage  people  to  openly  discuss  and 
solve  problems  that  affect  them.  This  means  that 
there  is  a safe  free  environment  created  with  no 
fear  of  management  retaliation.  People  enjoy  the 
freedom  of  expressing  their  thoughts  about  their 
work.  The  quality  circle  program  illustrates  how 
individual  employees  know  more  about  their  own  work 
than  anyone  else.  The  circle  members  also  enjoy 
getting  attention  and  help  from  engineering,  super- 
vision and  others  in  reducing  the  conflicts  they 
see  in  their  work  environment.  The  circle  program 
also  works  because  the  members  are  recognized  for 
their  successful  projects. 

Summary 

The  quality  circle  program  at  Hughes  Aircraft 
Company  illustrates  the  following  benefits  that 
contribute  to  MISSION  SUCCESS: 

1.  Improved  communications  between 
individuals  and  their  supervisor 
and  management. 

2.  Increased  interest  and  commitment 
members  have  in  their  job  and  in 
their  departments  performance. 

3.  Problem  identification  by  individuals 
that  really  know  the  problem. 
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4.  Solutions  to  problems  that  are  cost 
effect ive. 

In  other  words  quality  circles  increase  the 
utilization  of  information  and  resources  in  order 
to  enhance  MISSION  SUCCESS. 
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* From  an  8%  quality  return  rate  of 
products  by  aerospace/defense  customers  in 
1974  to  a present  day  rate  averaging  1-2% 
and  improving  further. 

* From  a situation  where  approximately  25 
quality  deficiency  reports  were  written  by 
government  source  inspectors  in  1974  to  a 
situation  of  an  average  of  only  one  per 
year  for  the  past  three  years. 

* From  a situation  where  corrective  action 
requests  were  handled  very  slowly  in  1974 
with  a backlog  of  over  100  requests  un- 
processed to  a no  backlog  situation  and 
immediate  action  today. 

* From  a situation  of  being  an  also  ran 
competitor  in  1974  to  having  the  largest 
number  of  JAN  qualified  products  of  any 
competitor  and  as  many,  or  more,  NASA  line 
certifications  today  as  any  competitor. 

All  of  this  has  happened  at  Signetics 
since  1974.  Why?  What  has  been  done?  It 
would  be  nice  to  be  able  to  point  to  one 
decision,  one  action  that  was  responsible 
for  these  changes.  However,  such  is  not 
the  case.  A continuing  series  of  decisions 
and  actions  were  required  and  were  built 
around  organizational  contingency  and  job 
design  theory  and  our  assessment  of  the 
implication  of  these  theories  for  our 
success  in  the  aerospace/defense  market. 

In  this  paper  I am  going  to  describe  a 
number  of  the  specific  actions  that  have 
been  taken  by  Signetics  Corporation  to 
structuring  and  organizing  the  company  and 
the  jobs  of  people  in  the  company  (across 
several  levels)  in  a manner  which  would 
maximize  our  ability  to  service  the  unique 
and  demanding  requirements  of  the  aero- 
space/defense industry.  Since  we  believe 
that  actions  taken  at  all  levels  have  con- 
tributed to  employee  motivation  and 
effectiveness,  I intend  to  describe  these 
actions  starting  from  the  top  management 
(corporate)  level,  then  proceed  to  look  at 
actions  taken  by  the  management  of  our 
Military  Products  division,  and  finally,  to 
review  some  specific  activities  to  improve 
performance,  motivation,  and  satisfaction 
at  the  operator  level. 

Corporate  Level  Actions 

Let  me  begin  the  discussion  of  specific 
Signetics'  actions  at  the  corporate  level. 
Much  of  the  guidance  to  our  thinking  about 
our  organization  and  management  practices 
comes  from  the  organizational  contingency 
theory  of  Lawrence  and  Lorsch.  The  theory 
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suggests  that  companies  operating  in  highly 
uncertain  and  complex  business  environ- 
ments such  as  high  technology  electronics 
need  to  adopt  organization  and  management 
practices  that  produce  both  a high  degree 
of  differentiation  (i.e.  much  specializa- 
tion of  function  and  substantial  differ - 
ences  among  the  orientations  and  thinking 
of  employees  in  the  specialized  functions) 
and  a complex  set  of  mechanisms  and  roles 
to  assure  effective  integration  of  the 
diverse  functions.  This  theory  has  pro- 
vided the  basis  for  much  of  the  company's 
effort  at  organization  development  (O.D.) 
and  improvement  since  the  formal  beginning 
of  the  O.D.  effort  under  the  then  new 
president  in  1971. 

In  1973,  Signetics'  management  realized 
that  the  nature  of  the  growth  in  the 
company's  business,  which  had  been  domin- 
ated by  growth  in  high  volume,  commercial 
product  lines,  had  generated  a need  for 
both  further  differentiation  and  special- 
ized integrating  structures  if  we  were  to 
succeed  and  improve  our  performance  in  the 
aerospace/defense  market  place.  The  in- 
creasing product  and  processing  complexity 
of  I.C.'s  and  their  relative  importance  to 
system  performance  was  creating  a need  to 
provide  more  intense,  dedicated  focus  on 
the  lower  volume,  but  more  demanding 
requirements  of  the  non-commercial  business . 
Simply  having  a hi-reliability  processing 
area  as  part  of  commercial  manufacturing 
was  not  sufficient.  Spec  reviews,  special 
qualifications,  quality  and  reliability 
assurance,  etc.,  needed  to  be  the  responsi- 
bility of  a group  of  people  whose  special- 
ized knowledge  and  primary  focus  was  on 
products  for  the  aerospace/defense  market. 

Therefore,  after  a series  of  discussions 
at  the  top  management  level,  we  made  the 
decision  to  restructure  a portion  of  the 
company  and  established  our  first  market 
oriented  (vs.  product/technology  oriented) 
division  which  is  called  the  Military 
Products  Division.  A division  manager  was 
appointed  and  given  P/L  responsibility  for 
the  business  as  well  as  his  own  dedicated 
facility.  Specialized  functional  resources 
were  pulled  together  to  be  integrated 
within  the  division  from  the  areas  of 
marketing,  finance,  special  (hi-rel)  manu- 
facturing and  testing,  engineering,  and 
production  control.  A specialized  military 
Q & R manager  was  appointed  who  reports  to 
the  corporate  QRA  manager  but  has  a dotted 
line  relationship  to  the  division  manager 
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and  functions  as  part  of  his  staff.  Within 
the  sales  department,  separate  groups  were 
established  to  focus  on,  manage  and  improve 
the  order  processing  and  customer  service 
functions,  an  industry  sales  manager's 
position  has  been  developed  to  coordinate 
field  selling  activities,  and,  where 
feasable,  geographically  dispersed  sales 
engineers  have  been  given  specialized  key 
account  assignments  and  made  part  of  the 
Military  Sales  team.  This  team  meets  semi- 
annually with  the  division  to  identify, 
confront  and  resolve  problems.  Addition- 
ally, the  assembly  operations  activity  in 
the  Orem,  Utah  plant  has  been  virtually 
entirely  devoted  to  processing  parts  to 
requirements  of  Military  specification 
38510. 

As  a result  of  these  changes,  the  com- 
pany achieved  a focus  on  the  business  that 
had  previously  been  impossible,  even  though 
many  of  the  resources  had  already  been 
spcifically  oriented  to  the  military 
business  in  the  previous  structure.  What 
had  previously  been  complex  problems  that 
might  go  unnoticed  or  unattended  in  the 
middle  of  other  problems  associated  with 
very  high  volume,  commercial  business , were 
not  only  noticed  and  attended  to  but  were 
the  top  priority  problems  of  the  people 
working  in  the  military  business  division. 
Common  problems  across  product  lines  were 
now  being  identified,  confronted,  and 
resolved  in  a coordinated  manner  by  mili- 
tary division  personnel  who  had  the 
legitimate  task  of  representing  the  company 
to  our  customers  with  our  salesforce  and 
integrating  activities  across  all  product 
lines . 

The  number  of  internal  interfaces  to  be 
managed  in  the  new  structure  was,  if  any- 
thing, increased.  However,  many  of  the 
needed  interactions  now  took  place  within 
the  division  where  common  overall  goals 
existed  and  where  most  of  the  people  in- 
volved were  located  together  within  the 
same  building.  Substantial  investments  of 
time  and  energy  were  made  in  developing  an 
effective  management  team  and  in  clarifying 
roles  and  working  relationships  among  the 
various  functional  groups.  For  nearly  a 
two  year  period  one  of  our  internal  organ- 
ization development  consultants  worked 
with  the  division  manager  and  his  organi- 
zation to  help  improve  their  effectiveness. 

Dealing  with  people  outside  of  the 
division  was  also  helped  greatly,  espe- 
cially in  the  areas  of  new  products  and 
service  to  customers,  by  the  development 
of  a separate  Profit/Loss  (P/L)  statement 
for  the  division.  The  product  division 
managers  had  always  had  worldwide  P/L 
responsibility.  Now,  however,  the  impact 
and  contribution  of  the  Military/Aerospace 
business  on  his  product  line  performance 
was  not  only  visible,  but  another  peer 
level  manager  shared  his  responsibility 


and  was  helping  him  to  improve  his  overall 
performance.  Changes  in  the  company's 
accounting  system  were  required  to  allow 
development  of  dual-counted  P/L  statements 
but  the  positive  effect  of  this  change  in 
the  measurement  system  (and  associated 
reward  systems)  was  immediate  and  obvious, 
especially  as  it  impacted  intergroup 
planning,  decision-making,  and  problem 
solving. 

Division  Level  Actions 


As  mentioned  earlier,  the  decision  was 
made  to  establish  a separate  division  and 
specialized  resources  in  key  functions  out- 
side the  division  in  order  to  achieve  an 
appropriate  level  of  differentiation  and 
focus  of  resources  and  to  build  a perman- 
ent business  structure  to  achieve  the 
integration  and  teamwork  necessary  for 
success.  The  division  management  team, over 
time,  found  it  necessary  in  some  areas  to 
provide  even  further  levels  of  differ- 
entiation and  specialization  which  in  turn 
required  additional  roles  and  mechanisms 
for  achieving  needed  integration.  For 
example,  as  time  progressed  the  organiza- 
tion began  to  "clean  up  its  act"  as  a 
result  of  the  increased  ability  to  concen- 
trate available  energy  on  the  tasks  of 
managing  the  business.  As  improvements 
were  made  and  some  basic  problems  solved, 
more  management  energy  could  be  focused  on 
growing  the  business.  One  area  of  parti- 
cular need  concerned  new  products,  both  in 
terms  of  new  designs/technology  and  in 
terms  of  bringing  viable  existing  products 
into  the  division.  To  accomplish  this, 
the  division  management  team  appointed  four 
business  teams  to  plan  and  coordinate 
activities  to  build  the  business  in  a 
specific  product  line.  Membership  on  the 
team  typically  included  representatives 
from  marketing,  specification  engineering, 
product  engineering,  production  control, 
manufacturing,  quality  and  reliability 
assurance,  and  product/design  engineering 
from  the  relevant  product  division.  These 
groups  are  charged  with  identifying  viable 
products  and  coordinating  their  energies 
to  develop  the  needed  capabilities,  cap- 
acities, and  logistics  to  bring  those 
products  to  the  military/aerospace  market- 
place . 

One  of  the  business  teams  is  focused  on 
an  area  of  particular  relevance  to  this 
conference  on  mission  assurance.  This 
group  is  called  the  Class  B Business  Team. 
Supplying  products  to  S-level  specifica- 
tions is  a complex  business  and  requires  a 
high  degree  of  intergroup  coordination  and 
communication.  Seeing  to  it  that  this 
coordination  takes  place  is  the  business 
team's  responsibility.  In  addition  to  use 
of  the  business  team  for  achieving  coordi- 
nation, the  division  has  also  appointed 
program  managers  to  assist  in  managing  the 
complex  logistics  and  status  requirements 
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of  their  customers.  They  too  are  part  of 
the  business  team. 

Operator  Level  Actions 

In  looking  at  actions  taken  at  the 
operator  level,  I'll  begin  by  continuing 
some  discussion  as  it  relates  to  operators 
working  on  S- level  products.  Success  in 
supplying  S-level  components  requires 
highly  trained,  experienced  operators 
certified  as  appropriately  skilled  for 
their  work  by  the  QRA  organization.  These 
operators  are  grouped  under  a production 
manager,  who  participates  in  the  business 
team,  whose  sole  focus  is  in  S-level 
processing  and  products.  This  group  was 
split  off  (another  degree  of  differentia- 
tion) from  a group  that  had  already  been 
set  up  to  process  newer  and  more  complex 
products  and  had  previously  split  from  more 
standard  processing  lines  to  give  it  more 
focus . 

One  concern  we  have  discussed  as  these 
sequential  splits  have  been  made  and  the 
work  of  the  production  lines  made  more 
specialized  is  that  we  avoid  becoming  so 
specialized  that  individual  jobs  become 
less  meaningful  and,  therefore,  motivating 
and  satisfying.  However,  to  date  we  have 
found  that  the  effect  ^f  the  changes  is 
more  in  the  direction  of  providing  oper- 
ators with  more  of  a small  work  team 
environment  focused  on  a portion  of  the 
business  with  which  they  can  identify. 

This  action  is  also  conslstant  with  other 
thrusts  in  the  company  aimed  at  assuring 
more  meaningful  and  motivating  jobs. 

First,  the  company,  with  thrust  provided 
by  the  President  himself,  is  moving  in  the 
direction  of  assuring  that  all  jobs  in  the 
company  are  designed  such  that  every 
employee  is  involved  in  the  management  of 
their  own  work.  Much  work  today,  partic- 
ularly at  lower  levels,  consists  primarily 
of  carrying  out  tasks  at  the  direction  of 
a supervisor,  i.e.,  "doing"  tasks.  The 
supervisor's  role  is  to  do  the  planning 
and  controlling  in  an  authority  oriented 
relationship  to  the  subordinate  whose  role 
is  to  respond  as  directed.  Arrangements 
like  these  produce  non-meaningf ul  work, 
low  motivation  and  untapped  capabilities 
and  energy. 

Supervisory  roles,  on  the  other  hand, 
viewed  as  involving  goal  oriented  relation- 
ships and  providing  assistance  to  sub- 
ordinates as  they  manage  their  work,  lead 
to  meaningful,  motivating  Jobs  that  tap 
the  energies  and  capabilities  of  all 
employees.  M.  Scott  Myers,  in  his  AMA 
book  "Every  Employee  a Manager",  covers 
this  concept  extensively.  We  feel  we  have 
a number  of  experiences  in  the  company 
demonstrating  the  practicality  of  such  an 
approach.  Probably  our  most  dramatic 


findings  occurred  a few  years  ago  in  one 
area  when  we  recorded  increases  in  line 
yield  (a  measure  of  quality  and  care  in 
handling  and  processing)  of  45%  in  a 4-week 
period  after  educating  and  informing 
operators  of  our  needs  and  objectives  and 
involving  them  in  the  planning  and  control 
of  their  work  to  achieve  those  objectives. 

The  concept  of  a supervisor's  role  as 
being  goal  oriented  and  assisting  goal 
oriented  operators  in  managing  and  carrying 
out  their  work  has  now  been  incorporated 
into  the  company's  supervisory  development 
program.  Supervisors  from  the  Military 
Products  Division  were  among  the  first  to 
be  exposed  to  the  program. 

Another  thrust  is  emerging  from  within 
the  division  towards  providing  more  mean- 
ingful , motivating  work  at  the  operator 
level.  Production  managers  became  aware 
of  an  experiment  with  the  use  of  a semi- 
autonomous  (self-managing)  work  team  on  an 
assembly  line  for  JAN  product  in  our  Orem, 
Utah  plant  (to  be  described  in  somewhat 
more  detail  below.)  Some  of  the  positive 
indications  from  the  work  in  Orem  lead 
them,  to  explore  several  places  where  the 
self-managing  team  concept  could  help  them. 
Their  conclusion  was  that  the  complexity 
and  generally  non-standard  nature  of  much 
of  the  Class  S product  processing  and 
testing  creates  conditions  where  the  flexi- 
bility and  motivating  potential  of  jobs  in 
a self-managed  work  team  could  produce 
both  improved  performance  (in  terms  of 
quality  and  improved  through-put  to  assure 
more  certain,  timely  delivery)  and  greater 
satisfaction.  Work  is  now  beginning  on 
the  evolution  and  development  of  the  semi- 
autonomous  team  concept  for  Class  S 
product.  Consultative  support  to  the 
effort  is  being  provided  by  the  Organiza- 
tion Development  Manager  of  the  Utah  plant, 
who  also  consulted  on  the  Orem  experiment, 
to  help  avoid  some  of  the  problems  en- 
countered in  Orem  and  to  bring  the  benefits 
of  the  learning  that  has  already  occurred. 

Orem  Self-Managing  Assembly  Team 

The  decision  to  engage  in  a work  re- 
design project  in  the  Orem  assembly 
operations  was  made  after  extensive  study, 
data  collection,  and  management  education 
and  discussion.  Indicators  that  work 
redesign  might  be  appropriate  were  high 
turnover  and  absenteeism,  frequent  pro- 
duction imbalance,  limited  cross-training 
of  operators,  and  verbal  inputs  indicating 
alienation,  morale  and  motivation  problems . 
The  Job  Diagnostic  Survey  developed  by 
J.  Richard  Hackman  of  Yale  University  was 
administered  to  production  operators. 
Response  patterns  indicated  job  designs 
with  low  motivating  potential  but  opera- 
tors with  generally  high  growth  needs, 
high  internal  work  motivation,  high  social 
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needs,  and  a high  desire  for  skill  variety, 
responsibility  and  challenge  in  their  work. 
This  pattern  supported  initial  thinking  in 
the  direction  of  a group  (vs.  individual) 
oriented  work  redesign.  A similar  pattern 
but  with  low  social  needs  would  have 
suggested  focus  on  individual  work  redesign. 

A design  team  was  formed  consisting  of 
an  area  supervisor,  a line  supervisor  who 
had  been  selected  to  be  the  team  advisor, 
a production  operator,  a quality  control 
operator,  an  engineering  technician,  and 
the  organization  development  consultant. 
Their  recommendations  were  accepted  and 
installed  in  a pilot  operation  in  mid-1977. 
About  four  months  later  a second  team  was 
established  on  the  night  shift  based  on 
encouraging  results  from  the  first  line. 

Some  of  the  changes  instituted  with  the 
team  line  are  as  follows: 

1)  Team  includes  a team  advisor  (vs.  line 
supervisor)  whose  role  is  advising,  con- 
sulting, and  liason  to  members. 

2)  Team  determines  line  capacity  each  week 
based  on  product  complexity,  scheduled 
vacations,  available  expertise,  etc. 

3)  Task  assignments  for  individuals  are 
decided  within  the  team. 

4)  Many  typical  maintenance  technician 
functions  are  performed  by  the  team 
members . 

5)  All  members  are  trained  and  certified 
as  quality  experts.  Fewer  external  checks 
and  use  of  QA  as  resource. 

6)  Cross-training  of  each  other  among 
members  - pay  goes  up  as  more  jobs  are 
mastered. 

7)  All  reports  and  information  needed  to  do 
the  Job  are  available  to  team  and  they  are 
taught  how  to  read  and  use  them. 

8)  Physical  layout  was  redone  to  facilitate 
relevant  Interactions. 

9)  Performance  is  evaluated  on  contribu- 
tion to  team  as  well  as  individual 
performance . 

Measuring  the  effects  of  these  and  other 
changes  is  difficult.  In  this  situation 
the  comparisons  that  have  been  made  are 
with  the  performance  of  three  other 
assembly  lines  running  similar  products. 
Over  a six-month  period  we  found  absen- 
teeism consistantly  below  the  area  average 
and  labor  efficiency  averaging  8%  above 
the  area's  performance.  Unit  costs  were 
not  apparently  different,  but  overhead 
efficiency  was  improved  substantially  as  a 
result  of  reductions  in  needed  overhead 
support  to  the  team,  such  as  supervisors, 
technicians,  and  production  aides  for 
training  and  helping  to  achieve  line 
balances . 

Of  particular  interest  to  this  mission 
assurance  conference  is  the  issue  of  the 
impact  of  the  work  redesign  on  quality 


performance.  It  is  in  this  area  that 
changes  of  the  type  described  have  typi- 
cally had  more  impact  (as  opposed  to  the 
area  of  productivity).  In  the  first  few' 
months,  the  data  we  had  indicated  that  the 
team  line  quality  was  below  that  of  the 
area  and  then  moved  quickly  to  a higher 
level  than  the  more  traditionally 
structured  lines.  Further  looking  into 
the  data,  however,  pointed  out  that  what 
had  happened  was  that  the  operators  on  the 
team  line,  given  full  quality  responsi- 
bility, had  responded  by  being  overly 
critical  and  rejecting  some  product  that 
the  QA  organization  would  have  accepted. 
This  did  not  change  until  assurance  and 
support  was  provided  by  the  QA  organiza- 
tion. 

Further,  the  initial  team  line  was  set 
up  on  the  day  shift  and  was  highly  visible. 
The  operators  responded  and  became  overly 
careful  in  their  inspections.  The  second 
line,  set  up  on  the  graveyard  shift,  where 
there  was  less  visibility  and  external 
pressure,  further  demonstrated  the  posi- 
tive impact  of  this  change  on  quality 
performance.  Quality  acceptance  rates 
began  at  very  high  levels  and  remained 
there.  The  levels  were  so  high  in  this 
case  that  additional  external  checks  were 
deemed  necessary  and  were  instituted  to 
verify  the  accuracy  of  the  operator's 
inspections.  The  conclusion  was  reached 
that  the  inspections  were  accurate  and 
that,  in  fact,  higher  qulaity  results  were 
achieved  than  we  had  previously  experienced . 

Summary 

This  paper  has  described  a variety  of 
changes  that  have  been  made  at  Signetics, 
from  the  corporate  level  to  the  operator 
level,  to  address  to  the  needs  of  the 
military/aerospace  market  and  the  needs  of 
our  employees  more  effectively.  In  virtu- 
ally every  case  the  result  of  the  changes 
have  created  more  motivationally  rich  jobs 
and  better  focus  on  the  tasks  involved 
leading  to  improved  performance  and,  there- 
fore, higher  levels  of  satisfaction.  From 
the  perspective  of  a parts  supplier,  we 
believe  that  the  effects  of  these  changes 
have  assisted,  and  will  continue  to  assist 
us  and  our  customers  with  the  task  of 
mission  assurance. 


Mailing  address: 

Dr.  Gerald  R.  Pieters 
Signetics  Corporation 
811  East  Arques  Avenue 
Sunnyvale,  California  94086 
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CARE 


W.  E.  Magnuson 

The  following  is  a brief  outline  of  the 
Convair  supplier  awareness  program: 

The  program  is  introduced  by  the  Manager  of 
Procurement  or  the  cognizant  Purchasing  Agent  who 
explains  the  business  outlook  for  Atlas/Centaur 
and  the  importance  of  our  suppliers'  contribution 
with  particular  emphasis  directed  towards  the 
group  we  are  talking  to.  A member  of  the  team  then 
describes  in  detail  the  use  of  the  equipment 
produced  by  that  particular  audience.  A color, 
sound  movie  "Countdown  to  Success"  is  then  shown. 

This  film  shows  some  of  the  accomplishments  of  the 
system  and  projects  what  will  happen  in  the  future. 

An  illustrated  lecture  called  "CARE"  follows 
in  which  the  listener  is  reminded  that  it  always 
costs  more  to  do  the  quality  job  the  wrong  way, 
that  it  costs  material,  time,  customer  dis- 
satisfaction; that  the  causes  for  doing  the  quality 
job  the  wrong  way  are  un familiarity,  failure  to 
follow  procedures,  doing  things  from  memory,  etc. 

The  need  for  an  error-free  environment  is  stressed. 
The  necessity  to  develop  personal  concern,  awareness, 
pride,  and  the  need  for  improvement  - that  personal 
concern  must  become  a habit  - is  emhasized;  that  in 
order  to  achive  this  goal  we  must  have  an  under- 
standing of  quality  related  objectives  and  the  new 
enviroment  that  surrounds  all  of  the  employees  in 
American  industry. 

The  employee's  part  in  the  products  produced  is 
emphasized.  The  absolute  importance  of  our  customer 
and  the  fact  that  since  each  of  us  uses  something 
from  someone  else,  we  all  are  in  ls^t  customers  is 
discussed.  It  is  then  pointed  out  that  everyone  in 
the  company  is  responsible  for  quality,  cost, 
schedule  and  new  business.  The  thesis  that  the 
whole  concept  of  CARE  requires  personal  commitment 
through  personal  awareness,  concern  and  enthusiasm 
is  developed. 

The  program  takes  about  b5  minutes . We  have 
given  it  approximately  150  times  to  approximately 
10,000  people.  Conservatively,  reception  has  been 
very  good  - in  fact  - the  usual  comment  is,  "Will 
you  come  back  and  give  the  same  message  to  all  of 
our  employees" . 


Mailing  address: 

Mr.  W.  E.  Magnuson 
General  Dynamics 
Kearny  Mesa  Plant  (KM) 

P.  0.  Box  8o81*7 
5001  Kearny  Villa  Road 
San  Diego,  California  92138 
MZ  1*1-11*60 

Mr.  W.  E.  Magnuson  is  a to  year  veteran  of  the 
aerospace  industry.  Beginning  his  career  at  the 
Douglas  Aircraft  Company  while  still  in  school. 
For  the  last  35  + years  first  with  Vultee 
Aircraft,  then  Consolidated  Vultee  or  Convair. 

He  has  held  a variety  of  management  positions  in 
experimental  aircraft  work  as  well  as  operation 
management.  Currently  is  a member  of  the  Quality 
Assurance  organization  at  Convair  in  San  Diego. 
Over  the  years  he  has  been  active  in  business 
affairs  including  National  President  of  the 
National  Management  Association.  A teacher  of 
Management  Discipline  in  the  San  Diego  Community 
College  System.  He  was  born  in  Minnesota, 
educated  in  Illinois  and  California.  He  and  his 
wife  Barbara  have  two  children,  Pamela  and  Dennis. 
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Motivational  Programs  at  Boeing  Aerospace 
Gene  Peretti 

KEY  WORDS:  Communication,  Motivation  Techniques, 
Productivity,  Crosstalk,  Problem 
Analysis,  Line  Management,  Satisfiers, 
Feedback,  Job  Related  Techniques 

The  general  feeling  about  Zero  Defects  some 
ten  years  ago  was,  it  was  one  of  the  things  we  all 
did  before  we  really  knew  anything  about  the 
science  of  motivating  employees. 

Since  Zero  Defects  we  have  experienced 
Satisfiers  and  Dis  Satisfiers.  We've  identified 
X and  Y managers,  we've  conducted  surveys,  we've 
graded  and  classified  groups  of  people  - all  in 
an  effort  to  teach  us  something  about  people  at 
work.  The  only  difficulty  with  most  of  this  new 
knowledge  is  that  such  techniques  are  very  difficult 
to  communicate  to  Line  Management  and  almost  im- 
possible to  communicate  to  the  troops  - so  the 
experts  end  up  talking  to  themselves.  As  bad  as 
Zero  Defects  was  in  the  eyes  of  most  behavioral 
scientists,  it  was  one  program  that  everyone  knew 
about  and  that  most  people  understood.  I'm  not 
suggesting  a return  to  Zero  Defects,  but  there's 
a lesson  here,  if  we  can't  come  up  with  an  object- 
ive that  is  clearly  understood  by  every  manager 
involved  and  that  can  be  just  as  clearly  comnuni- 
cated  to  everyone  on  the  program,  we’re  only  fool- 
ing ourselves.  No  improvement  will  be  made,  ob- 
jectives and  targets  will  be  ignored,  missile  hard- 
ware will  fail  and  line  rejections  will  continue  to 
pile  up  unless  the  specific  goal  and  objectives 
are  clearly  conmunicated  to  every  employee. 

Chart  1 

o The  Boeing  Industrial  Relations  role  in  product- 
ivity has  to  do  with  the  effective  utilization 
of  the  work  force. 

o Obviously,  capital  investment  has  traditionally 
been  the  big  swinger  in  improved  productivity. 

I would  estimate  that  the  people  impact  on 
productivity  in  BAC  (Boeing  Aerospace  Company) 
is  something  like  20%  to  30%  as  compared  with 
70%  to  80%  for  capital  investment  in  new 
machines. 

o We  should  not  underestimate,  however,  the 
importance  of  people  as  a resource.  If  we  do 
our  job  right,  the  result  can  have  a substantial 
Impact  on  company  profit. 
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■ 


Chart  2 

As  shown  here,  we  get  a lot  of  help  from  our  Corpor- 
ate Headquarters  in  terms  of  employee  related  bene- 
fits. The  effect  that  such  benefits  have  on  the 
"bottom  line"  is  sometimes  difficult  to  measure, 
but  without  this  base  we  would  be  hard  pressed  to 
build  workable  improvement  programs  to  increase 
productivity.  You've  got  to  provide  the  basics 
first,  before  you  can  motivate  people. 

Corporate 


• Company  ten  ire  award* 

• Booing  news  (weekly) 

• Rc  creation  programs 

• Club  memberships 

• Tennis  flub 

• Picnic  kill 

• J jiltiii;  Derby 

• Dofuunt  tickets 

• Trawl  packages 
Q Co or.  prog:«.r.t 
O Sl.jrc-tl.c-Rivh?  program 

• Retirement  Planning  Program 
C Boeing  Employee*  Good  Neighbor  Fund 

• Boeing  Employee*  Disaster  Relief  Fund 

Chart  3 

In  setting  the  stage  for  a discussion  of  BAC  Product- 
ivity programs,  I feel  it's  vital  to  relate  such 
programs  to  the  unique  characteristics  that  exist 
in  BAC.  We're  a "job  shop"  with  a highly  diversi- 
fied product  mix,  that  usually  results  in  contract 
runs  of  less  than  twenty  items. 

With  short  runs,  it's  difficult  to  make  an  exper- 
ience curve  pay  off,  you  have  to  move  people  from 
program  to  program  and  you're  dealing  with  a con- 
stant retraining  process.  All  of  this  can  cost  a 
lot  of  money  and  our  job  is  to  minimize  such  costs 
and  to  develop  the  kind  of  people  that  can  be  pro- 
ductive in  this  kind  of  business  environment. 


• Employee  Christum  Party 

• Employee  benefit  plain 

• Insurance  packages  • Vacation 

• Voluntary  Investment  Plan  * Sick  leave 

• Financial  Security  Plan  • Retirement  Plan 

• Management  Information  Bulletin  (daily) 

C Manner  baseball 

fr  hafihtics  modcrnitr.tion 
C Employee  awards  policies 

C Community  contribution*  (fji.Jiiiet.  money,  lime) 

® Employ  ee  Scli-Devclopmcnt  Program 


Our  approach  to  productivity  through  people  is 
designed  to  meet  the  unique  needs  of  each  organiza- 
tion and  it's  product  line  and  employee  mix. 

Boeing  Aerospace  Company  Approach 


O Recognize  unique  BAC  features: 

• 45  to  55  named  programs  (ongoing) 

• $0rr  of  deliveries  Stave  quantity  of  less  than 
20  items 

• Items  from  heavy  steel  to  microelectronics 

• Matty  types  of  customers 

• Leads  to  a diversified  organization 
(nor.  traditional) 

o Develop  diverse  motivation  programs  to  accommodate: 

• Organizational  structure  and  skill  mix 

• Management  style  of  leader 

• Prime  program  objectives 

• Feedback  to  employees 

• Focus  on  BAC  as  a good  place  to  work: 

• Tcrsonalizcd  approach 

• Challenge  of  new  programs 
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Chart  4 


Before  I get  into  specific  techniques.  I'd  like  to 
point  to  some  of  the  “people"  problems  that  deter 
productivity  and  impact  quality  that  can  influence 
improvement  in  these  areas  if  we  know  what  the 
facts  are  and  if  we  can  get  management  to  cooperate 
with  us  in  applying  a fix.  It  would  be  interesting 
to  know  how  you  would  rate  these  nine  items  in 
order  of  importance.  A recent  study  of  BAC  Opera- 
tions management  placed  negative  management  atti- 
tudes as  number  one  and  low  employee  morale  as 
number  two.  I'm  going  to  talk  about  these  items 
at  the  end  of  the  pitch  under  impediments. 

Deterrents  to  Productivity 


Chart  5 


Chart  6 (cont. ) 

As  shown  here,  these  are  job-related  techniques. 

What  Is  the  Objective  ? 


Chart  7 

This  is  an  example  of  work  environment  techniques. 
Such  improvements  are  necessary  and  should  be  made. 
However,  in  terms  of  productivity  and  quality  they 
are  sometimes  difficult  to  measure. 


Let's  look  at  "people"  techniques.  This  is  not  a 
complete  list,  but  it  is  fairly  representative  of 
the  wide  range  of  specialized  techniques  that  have 
been  in  use  in  BAC  for  some  time. 

Our  approach  has  been  to  work  with  each  principle 
BAC  manager  to  determine  his  approach,  needs  and 
management  style,  and  then  to  tailor  a program  for 
him  using  these  techniques  as  tools. 


Chart  6 

In  applying  the  techniques  (shown  on  the  previous 
chart),  we  point  out  to  Line  managers  the  differ- 
ence between  techniques  that  Impact  profit  (sche- 
dule, cost  and  quality)  and  others  that  affect  work 
environment  or  the  family  and  community. 


Chart  8 


Even  further  removed  from  profit,  but  nevertheless 
helpful,  are  such  items  as  golf  tournaments,  tennis 


Chart  8 (cont. ) 


Currc.il  activities: 


Chart  9 

This  illustrates  the  sequence  of  events  of  a typical 
people  program  in  one  of  our  BAC  organizations. 

As  shown,  we  attempt  to  relate  everything  we  do  to 
the  organization's  business  objectives.  That  pro- 
cess helps  to  assure  that  profit  and  productivity 
will  be  impacted  and  also  that  we  get  the  top 
manager's  support.  He  is  obviously  going  to  be 
behind  a program  that  directly  impacts  his  bottom 
line  and  his  incentive. 


Motivation  Program  Planning  Cycle 


Chart  10 

This  chart  shows  the  individual  programs  and  t » 
fact  that  all  of  the  major  organizations  have  p. . **t- 
iclpated  with  us  in  the  development  of  a tailored 
program. 

Each  of  these  organizations  has  a specialist  ap- 
pointed p *t  time  or  full  time  to  make  sure  the 
program  ytts  caimunlcated  and  that  it  produces  re- 
sults. 


Chart  10  (cont, ) 

New  and  Revised  Motivation  Programs  for  1977 
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Chart  11 

By  way  of  illustration,  here’s  the  kind  of  Wish 
List  that  usually  comes  out  of  the  early  meetings 
with  management. 


Roland  Motivation  “Wish  List’ 


• Inside  parking  for  all  salaiicd  employees. 

• Space  is  available  to  accommodate  people. 

• 1R  anti  Security  approval  required. 

• Dedicate  1 equivalent  BAC  (tip  award  (S4.C0O  value)  to 
Roland  program 

• Use  to  fund  approximately  10  trips  for  two  to 

Disneyland 

° .utbly  Itmcheon  with  program  manager  for  team  demonstrating 
r.c«w  .••liv.v.v.ant  on  recove . y targets  (with  Selby  and 
Keister,  as  necessary). 

• Roving  clock/piaquc  award  presented  to  monthly  team  achievers. 

Chart  12 

The  Employee  Service  centers  that  have  been  set  up 
at  Plant  II,  Kent  and  Development  Center  are  another 
example  of  the  kind  of  technique  that  can  have  a 
positive  impact  on  employee  self-development  and 
employee  participation  in  company  sponsored 
programs. 
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Chart  12  (cont.] 


Chart  14  (cont. ) 


E9twu  si  .'las  cr;.TtR 

l aw"  . . s<:  ..:ce:  fw  km  «ir:  t ••  *i:.v£W 

r kiwis 

c-.  *Lli.v.  voih 

sfej  ti'uT  tViir* 

i.:  i-;*i -r.z  ikuc  tax  ?K"5 

15i*a  KW-tAL  A .-  fWILinc:  — U AttU  CE51U 
■ j v 

n.  ir.a'-:; 

Sp/-i.vm 

i .:.\c:u 

v;;.n.  \.yn  ••.:  r.:i«Cs,T 

fc*l>  : fc-.l  : A 1 ' •'  l*.U  C* 

Shw?  CSii  .££!»:. jS 

lAIiJ  • .!*•  fACllill  . ' WjjA  USttt 

«r.  >;:u  a?!*  kv».  .t*t  • 


Chart  13 

A new  effort  that  grew  out  of  the  Roland  cost  pro- 
blem is  our  Industrial  Relations  sponsored  Crosstalk 
Breakfasts. 

The  way  we're  organized  in  BAC,  there  is  a critical 
need  for  close  and  effective  comnunication  between 
each  program  manager  and  all  levels  of  the  support 
organizations. 

Objective 

Crosstalk  brings  the  lower  level  managers  into  a 
meeting  with  Program  managers  that  would  not  other- 
wise take  place.  It's  also  a recognition  experience 
for  lower  level  managers  so  selected. 
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Chart  14 


These  sessions  also  Involve  Finance,  Industrial 
Relations,  Materiel,  Quality  Assurance  and  other 
such  support  people. 


WHAT  IS  A CROSSTALK  BAEV', AST? 

C IHt  PURPOSE  OF  A CF.OSS'. ALn  BRLAKF AST  IS  TO  SR i I.G-PERSOHS 
INVOLVED  WITH  A SPECIFIC  PkOGRA*!  TOGETHER  Hi  A RELAXED 
ASS  INFORMAL  ATMOSPHERE,  TO  REVIEW  REV  PROGRAM  OBJECTIVES 
AND  POTENTIAL  PRC&iK  AREAS,  AND  TO  DISCUSS  WAVS  TO  AVOID 
THOSE  PROBLEMS, 

o INVOLVED  PERSONS  ARE  THOSE  WHO  CREATE  THE  WORK  TO  BE  DONE 
TPhuCSAM  PtKST  . .EL),  AND  THOSE  WHO  30  THAT  WORK  (SUPPORT 
CRy.MZ.AI  ION  PERSOII'IEE) . 


Chart  15 

The  objectives  here  are  program  related  and  often 
involve  discussions  of  cost,  schedule  and  quality-- 
the  importance  of  holding  cost  in  line  with  program 
cownitments,  the  impact  of  overruns  on  continued 
business  and  future  sales,  the  long  term  impact  of 
current  performance  on  future  business. 


OBJECTIVES 


O ESTABLISH  CLOSE.!  COOU'iNAi I. .VS  IMPROVE 
WORKING  RELATIONSHIPS  BETWEEN  SUPPORT  ORGAN- 
IZATIONS AND  ONGOING  PROGRAMS  IN  A LESS 
rCRTAu  SETTING. 

f\  ALLOW  I'ROARA'l  AND  SI-SPORT  SMPFIWISOK  THE 
OPPORTUNITY  TO  MEET  ,'J.j  TACK  !:1  EACH  OUR. 
AND  in  A FSiGRAM  WSACcR  ON  PROGRAM  PI  SUETS 
MUCH  INVOLVE  SCHEDULES,  804LITY,  DELIVERIES, 
ETC. 

Q CREATE  A GREATER  BE GHEE  OF  TEAM  SPIRIT, 
COOPERATION  AND  UNDEKSTAUDIIIC. 

O •RECOGNIZE  THOSE  FIRST  LEVEL  SUPERVISORS  WhO 
DEMONSTRATE  COST  KLCUCTION  .AND  OPERATIONAL 
IMPROVEMENTS. 

O INVITE  CUSTOMERS  AS  APPROPRIATE  ANE  WHEN 
AVAILABLE. 


Chart  16 

The  organizations  shown  on  the  left  side  of  this 
chart  are  Harry  Goldie’s  support  team  and  the  pro- 
grams on  the  right  are  the  prime  business  opport- 
unities for  BAC. 

Without  clear  lines  of  comnunication  between  pro- 
grams and  support  organizations,  the  unique  and 
sometimes  tough  performance  cornni tments  of  these 
programs  may  not  be  achieved. 
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Chart  16  (cont.) 


Chart  19 


WHO  IS  INVOLVED? 


SUPri.'.'T  PERSONNEL 


PROGRAM  PERSONNEL 


CC.iT  SACTS 
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MANUFACTURING 

MATERIEL 
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ELECTRONICS  SUPPORT 
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Chart  17 


As  shown  on  this  chart,  PAC  is  an  improvement  on 
PEPS.  The  basic  change  is  the  involvement  of  the 
Line  Supervisor  as  a facilitator  instead  of  the 
Industrial  Relations  specialist. 

The  problem  with  PEPS  was  when  the  Industrial 
Relations  facilitator  left  the  group,  the  group 
would  soon  fold  up  for  lack  of  leadership.  Also, 
the  Line  supervisors  objected  to  their  people  going 
to  PEPS  meetings  without  them. 


ADVANTAGES  OF  PAC 


• LESS  COMPLICATED  THAN  PE’S 
•TORE  DIRECT  - NO  I.R.  INTERVENTION 

• I.R.  INVOLVED  ONLY  IN  TRAINING 

•LINE  SUPERVISOR  DIRECTLY  INVOLVED  AS  FACILITATOR  AND  KEM3ER 

OF  COOPERATIVE 

• IMPROVED  COORDINATION  AND  FEEDBACK 


We've  held  three  breakfasts  over  the  last  six  months 
and  plan  three  more  before  year  end. 

The  last  breadfast  was  video  taped  and  features 
the  Roland  program  with  Jerry  King  — WxT're  going 
to  run  that  television  show  for  all  shop  and  office 
areas  involved  with  Roland. 

I've  supplied  cups  and  pins  to  you.  These  are 
inexpensive  items  that  the  first  line  guys  can  take 
back  to  work  and  demonstrate  that  they  were  select- 
ed to  participate. 


•SUCCESSFUL  OPERATION  DIRECTLY  ATTRIBUTABLE  TO  ORGANIZATION 
•ACHIEVEMENTS  TIED  DIRECTLY  INTO  ORGANIZATION  GOALS 

Chart  20 

PAC  then  is  a job-related  problem  solving  technique 
that  is  aimed  ft  improving  productivity  by  "working 
smarter" 

PAC  IS  


A SYSTEM  WHICH  RECOGNIZES  AND  UTILIZES  THE  ABILITY 
OF  EMPLOYEES  TO  IDENTIFY  AND  SOLVE  PROBLEMS  PI  RECTI  V 


r<> 

w 

NEED  TO  FIND  TF..IER  1 lYS  10  CC.MMUNICAIE 

BETWEEN  IIZAT!  V - .. 

ON.jCTNG  . . . 

Chart  21 

0 

TEED  TO  STIMULATE  INTEFiEST  AND  SUPPORT 

AT  SUPERVISORY  LEVEL. 

Problems  are  surfaced  by  employees  and  solutions 
for  resolution  are  offered  by  the  group  using  a 

TEED  10  PROVIDE  CROSS -FiiNCi  1 ORAL  RECOG- 
NITION OF  INDIVIDUALS  WHO  PROVIDE  SUPPORT 

problem  solving  technique. 

TO  EACH  OTHER. 

METHOD  OF  OPERA! ION 

o 

CEO  TO  EXPOSE  SHOP  MD  OFFICE  SUPERVISORS 

TO  PROGRAM.  MAIWER'S  OBJECTIVES. 

A GROUP  OF  6-8  VOLUNTEER  EMPLOYEES  WILL  KEEI 

...3  TP  TEAS  SPIRIT  THROUGH  BETTER 

WITH  THEIR  SUPERVISOR  TO  ANALYZE  A WORK-RELATED 

c CAT  IMS.  problem  and  arrive  at  a solution  to  that  problem 


Chart  18 

A new  program  that  we  plan  to  kick  off  this  fall 
is  called  "PAC"  for  "Problem  Analysis  Cooperative". 
It's  an  off-shoot  of  PEPS,  the  Participative 
Employee  Problem  Solving  program  that  was  In  oper- 
ation for  about  three  or  four  years. 


Chart  22 

The  group  size  is  limited  to  6 - 8 volunteer  employ- 
ees who  meet  with  their  supervisor.  The  fact  that 
employees  volunteer,  helps  to  assure  that  Interested 
people  will  be  involved  on  teams. 


PAC 

PR03IEM 

ANALYSIS 

COOPERATIVE 


• . > 

- •>.  - 
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Chart  23 


This  flow  chart  Illustrates  the  primary  steps  that 
take  place  in  setting  up  a typical  problem  solving 
team. 


In  PAC  we  plan  to  formalize  our  Supervisor  training 
in  problem  solving  to  include  a comprehensive  course 
on  brainstorming,  group  dynamics  and  how  to  work 
with  a group  rather  than  give  directions.  Such 
training  will  be  recorded  in  the  supervisor's  train- 
ing record  and  his  success  with  his  group  will  be 
a plus  factor  in  his  rating. 

See  above  chart  for  reference. 


Chart  24 


The  output  of  PAC  groups  is  primarily  to  solve  pro- 
blems but  as  shown  here,  there  are  a number  of  side 
benefits  worth  noting.  It's  also  worth  noting  that 
the  Suggestion  System  doesn't  surface  all  of  the~~ 
.job  related  problems.  Many  problems  are  too  com- 
plex for  suggestions,  or  employees  have  difficulty 
getting  their  ideas  down  on  paper. 


pac  benefits 


• IDENTIFIABLE  DOLLAR  SAVINGS 


• EMPLOYEE  SATISPACTIOII 

• INCREASED  PRODUCTIVITY 

• TETTER  UNDERSTANDING  CF  COMPANY  COALS 

• INCREASED  EMPATHY 

• IMPROVED  COtUMUCAIIOH  SKILLS 
•3E7TER  PEER  RELATIONSHIPS 

• HEIGHTENED  AWARENESS  OF  COST  IMPACT 


Chart  25 


To  sum  up  --  the  supervisor  leads  the  group  and 
gets  recognition  for  his  training  and  success  with 
problem  solving. 


One  final  point  --  the  management  from  the  top 
down  must  be  comnitted  to  the  program  and  demon- 
strate that  Comal tment  by  encouraging  lower  level 
managers  to  work  with  groups  and  seek  better  ways 


Chart  25  (cont.) 

CRITERIA  FOR  S'.rfSS 

• SUPERVISOR  IS  A MEMBER  OF  CO-OP 

• MANAGEMENT  COMMITMENT  IS  MANDATORY 

• ACHIEVEMENTS  MUST  EE  RECOGNIZED 

• COMPANY-WIDE  PUBLICITY 

• EMPLOYEES  IDENTIFY  PROLEEKS  AND  ARRIVE  AT  SOLUTIONS 

• PRODLEMS  MOST  EE  CURRENT  AND  WORK-RELATED 

• CO-CP  MEMBERSHIP  MUST  BE  ROTATED 
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Management  Motivation  and  Contractual  Incentives 
David  R.  Stevenson 
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Abstract 

This  paper  is  an  abbreviated  descrip- 
tion of  the  Honeywell  Avionics  Division, 

St.  Petersburg,  Florida  location,  training 
and  motivation  programs. 

I n t roduct ion 

Honeywell  St.  Petersburg  has  com- 
prehensive training  and  motivation  pro- 
grams directed  to  assuring  that  personnel 
are  qualified  and  understand  the  need  for 
high  quality,  cost  effective  performance 
to  assure  contractual  requirements  are 
met.  The  programs  assure  that  engineering, 
operating  and  supervisory  personnel  are 
trained,  qualified,  and  where  required, 
certified  to  customer  requirements. 

Training 

Operating  Personnel 

Newly-hired  operating  personnel  are 
required  to  take  AO  hours  of  classroom 
instruction  in  soldering  and  be  certified 
prior  to  performing  a production  or  in- 
spection function.  They  must  produce 
samples  demonstrating  soldering  capability 
prior  to  being  certified.  Certification 
is  granted  by  the  Central  Training  School. 
Upon  assignment  to  the  function,  these 
personnel  are  given  vestibule  and  on-the- 
job  training  in  the  particular  soldering 
techniques  they  will  use  to  produce  con- 
tractually acceptable  product. 

In  addition  to  soldering,  personnel 
who  will  be  performing  plastics  operations- 
e.g.,  product  encapsulation,  conformal 
coating  - are  given  a kO  hour  classroom 
course  in  mixing  and  applying  plastics. 

As  with  soldering,  they  must  make  samples 
to  demonstrate  their  skills  prior  to  being 
certified  by  the  Centra)  Training  School. 
They  are  also  given  vestibule  and  on-the- 
job  training  prior  to  working  on  the 
product . 

Personnel  who  will  perform  resistance 
welding  are  certified  in  a similar  manner 
as  those  certified  for  soldering  and 
plastics. 

Special  processes  - e.g.,  welding, 
heat  treat,  plating,  plated  wire  processes- 
are  taught  in  vestibule  and  on-the-job 
training  programs.  Personnel  performing 
these  processes  are  certified  only  after 
acceptable  demonstration  samples  are 
comp  1 e ted . 

Certifications  are  rescinded  if  a 
person  is  not  performing  the  function  for 
which  certified,  workmanship  falls  below 
accepted  standards,  or  specified  time 
Intervals  for  rece r t f f I ca 1 1 on  have  lapsed. 
Records  are  maintained  on  each  person 
and  are  under  continuing  assessment  to 


assure  that  their  certifications  are  not 
expired  and  their  performance  remains  up 
to  goals. 

Systems  Personnel 

College  level  classes  are  available 
at  several  universities  and  colleges  in 
the  immediate  St.  Petersburg  area  for 
engineers  to  update  their  technological 
backgrounds.  In  addition,  in-house 
courses  in  current  technologies  are 
taught  by  university  instructors  and 
professors,  and  qualified  Honeywell 
personne 1 . 

Test  technicians  are  required  to 
have  completed  and  graduated  from  an  ac- 
credited technical  school  prior  to  being 
hired.  College  and  technical  institute 
courses  are  available  to  these  personnel 
to  update  their  knowledge  to  new  tech- 
nologies. In-house  courses  are  also 
taught  to  cover  special  technologies  and 
unique  program  characteristics. 

Product  Logistics  Support  Personnel 

Logistics  Support  personnel  operate 
In  the  field.  They  are  engineering  level 
personnel,  supported  with  site  managers 
on  major  programs.  In  addition  to  their 
college  educations,  they  are  given  in- 
house  and  site  training  In  the  unique 
requirements  of  the  programs  to  which  they 
are  assigned.  Courses  for  this  training 
are  documented  in  a Training  Services 
Course  Catalog.  Correspondence  training 
is  also  available  to  personnel  while  In 
their  field  locations. 

Incentive  for  Self-Improvement 

All  personnel  are  entitled  to  tuition 
aid  f or  Job  - re  1 a t ed  training  at  approved 
colleges,  universities  or  technical  In- 
stitutes. Reimbursement  under  the  pro- 
gram Is  based  on  achievement  - i.e., 
grades  or  marks  received  - for  each 
course  completed. 

Motivation 

Motivation  programs  in  place  Include 
a d I v I s I on-w i de  STEP  (Strive  Toward 
Error-free  Performance)  program,  periodic 
communications  sessions  at  all  levels  In 
the  organization,  recognition  awards  for 
length  of  service,  and  program  manager 
meetings  with  program  personnel  to  ap- 
prise them  of  goals  and  accomplishments, 
problems,  and  future  projections. 

Incentive  Programs 

the  STEP  program  recognizes  In- 
dividual contributions  to  design,  process, 
operational  and  cost  Improvements.  Re- 
cognition Is  given  in  award  meetings  and 
via  company  newspapers.  Awards  can  go 
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up  to  several  hundred  dollars  in  savings 
bonds  . 

A Quality  Awareness  Program  has  been 
Instituted  to  recognize  operating  per- 
sonnel and  supporting  groups  for  achieving 
product  yield  and  defects  per  unit  goals. 
Awards  include  luncheons  for  short  range 
accomplishments,  and  monetary  recognition 
for  long  range  accomplishments. 

Service  Award  Luncheons  and  pub- 
licity in  Honeywell  newsletters  are  a 
continuing  feature  of  our  motivation 
efforts.  Recognition  is  given  not  only 
for  length  of  service,  but  also  for  out- 
standing accomplishments. 

Commun i ca  t ions 

Communications  meetings  are  held  by 
top  management  for  all  supervisory  per- 
sonnel twice  yearly.  Quarterly  department 
director  meetings  are  held  with  all  levels 
of  personnel  in  the  respective  department. 
Supplementing  these  are  weekly  meetings 
between  directors  and  their  staffs,  and 
monthly  meetings  by  line  supervisors  with 
their  hourly  and  salary  personnel.  All 
communications  meetings  are  directed 
toward  achieving  face-to-face  dialogue 
on  problems,  questions,  and  projections. 
These  meetings  have  been  highly  success- 
ful in  creating  a unified  drive  toward 
the  objectives  of  customers  and  Honeywell 
management . 

Program  Managers  hold  meetings  as 
appropriate  to  enlist  emphasis  on  solving 
problems,  to  give  recognition  for  achieve- 
ment, discuss  business  projections  and 
other  information  related  to  program 
goals.  Astronaut  visits,  customer  visits, 
launch  visits,  and  customer  furnished 
films  are  a part  of  this  motivational 
effort . 

Measurement  of  Effectiveness 

Overall  effectivity  is  reported  in 
a computerized  Quality  Measurement  System 
(QMS-200)  and  computerized  schedule/cost 
reports.  QMS-200  provides  yield  and 
defects  per  unit  by  program,  work  centers, 
operator  and  process.  The  system  can  be 
interrogated  for  any  element  contained  in 
the  history  file  - e.g.,  part  number, 
assembly  number,  problem  definition 
(cause/defect  code),  etc.  Schedule/cost 
reports  show  schedule  and  cost  positions 
by  project,  charge  number  and  supervisor. 
It  provides  for  daily  input  and  output, 
produces  periodic  summaries  and  Is  a 
subject  of  periodic  top  management  review. 

Summary 

Honeywell  St.  Petersburg's  training 
and  motivation  programs  begin  with  new 
hires  and  are  continuing  elements  In 
employees'  job  assignments.  They  provide 
Incentive,  awards,  review  and  corrective 
measures.  The  programs  are  baselined  to 
d I v I s I on-w I de  policies  and  provide  con- 
tract Incentive  awareness  for  each  program 
in-house,  as  well  as  anticipated  new 
programs . 


Mailing  Address: 

Mr.  David  R.  Stevenson 
Honeywe II,  Inc. 

13350  U.S.  Highway  19 

St.  Petersburg,  FI.  33733  USA 

David  R.  Stevenson  was  born  in  Kansas  City, 
Missouri.  He  attended  Kansas  University, 
Metropolitan  College  in  Kansas  City  and 
St.  Petersburg  Florida  Junior  College.  He 
became  associated  with  the  Kansas  City 
Division  of  Bendix  Corporation  In  1950 
and  was  instrumental  in  initial  develop- 
ment of  product  test  facilities  and 
es t ab 1 i s hmen t of  a total  quality  assurance 
operation.  He  is  currently  Manager  of 
Process  Quality  Assurance  at  Honeywell, 

St.  Petersburg,  Florida,  responsible  for 
Quality  Engineering,  final  product  accept- 
ance, process  development  and  support. 
Acceptance  Testing  and  Inspection  opera- 
tions. He  has  participated  in  a number 
of  quality  seminars  throughout  the 
country. 


Training/Motivation  Goals 

• Workshop 

• Define  SAMSO  counter  production 
practices  and  philosophies 

• Provide  effective  mission  assurance 
related  recommendations 

• Define  recommended  standards  of 
operation  to  SAMSO 

• Define  need  for  on-going  workshop 
sessions 

• Honeywell 

• Provide  a training/motivation 
program  that  is  cost-effective, 
performance  oriented,  and  supportive 
of  mission  effectiveness 

• Systems  Personnel 

• College  level  education 

• Technology  updates 

• In/out-of-house  training 

• Program  familiarization 

• Program  orientations 

t System  mission/effectiveness 

• Testing  and  test  equipment 

• Accredited  technical  schools 

• Test  managers  - engineers 
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PRODUCT  ASSURANCE  TRAINING  l ORIENTATION  PROGRAM 

SUBJECT 

LEAD  DEPT.  (P.A.) 

DURATION 

f\A, 

, SPECIFICATIONS 

P.A.  PROGRAMS 

1. 

INTRO  TO  SPEC  TREE 

P.A.  PROGRAMS 

1.5  HRS. 

2. 

OoD  QUALITY  SPECS.  45208  t 9858 

P.A.  PROGRAMS 

1.5  HRS. 

3. 

CAL  I BRAT lOM/METROLOGY  45662 

PROCESS  Q.A. 

1.0  HR. 

4. 

NON-COtf.  MAT'L.  f.  CORR.  ACT.  1520 

CENTRAL  SERVICES 

1.0  HR. 

5. 

PROCUREMENT  QUALITY  1535 

CENTRAL  SERVICES 

1.0  HR. 

6. 

FBM  P.A.  REQUIREMENTS  21549 

P.A.  PROGRAMS 

1.0  HR. 

7. 

NASA  P.A.  SPECS  (5300  SERIES) 

P.A.  PROGRAMS 

2.0  HRS. 

P.A.  PROGRAM  PLANNING  t MANAGEMENT 

8. 

INTRO  TO  PROGRAM  PLANNING 

P,A.  PROGRAMS 

1.0  HR. 

9. 

IMPLEMENTATION  OF  PROGRAMS 

P.A.  PROGRAMS 

1.0  HR. 

10. 

PROGRAM  REPORTING  1 ASSESSMENT 

CENTRAL  SERVICES 

1.0  HR. 

PA 

. POLICIES  & PROCEDURES  - U-ED  23036 

11. 

ADMINISTRATIVE  POLICIES  8 

P.A.  PROGRAMS 

1.5  HRS. 

PROCEDURES  (A  ,B,  C) 

12. 

SECTION  D - QUALITY  SYSTEM  - D1-D3 

P.A.  PROGRAMS 

1.0  HR. 

13. 

D4.1  - D4.5 

PROCESS  O.A. 

1.5  HRS. 

14. 

D4.6  - 04.13 

PROCESS  Q.A. 

1.0  HR. 

P.A.  POLICIES  8 PROCEDURES  - U-EC  23036  (COIIT'D) 


15. 

D5-D10,  D15,  D15.1,  D16 

CENTRAL  SERVICES 

2.0  HRS. 

16. 

Dll 

PROCESS  O.A. 

2.0  HRS. 

17. 

D12 

CENTRAL  SERVICES 

2.0  HRS. 

18. 

D17  - D20 

CENTRAL  SERVICES 

1.5  HRS. 

19. 

D21,  D26 

MATERIEL  O.A. 

1.5  HRS. 

19A 

D25 

COMPONENTS  P.A. 

1.0  HR. 

COST  OF  QUALITY 

20. 

COST  OF  QUALITY  CONCEPTS 

CENTRAL  SERVICES 

1.0  HR. 

21. 

COST  OF  QUALITY  HONEYWELL  APPROACH 

CENTRAL  SERVICES 

1.0  HR. 

22. 

WORKMANSHIP  STANDARDS 

CENTRAL  SERVICES 

1.0  HR. 

23. 

SOFTWARE 

MATERIEL  Q.A. 

1.0  HR. 

TOTAL  QUALITY  CONTROL 

PROCESS  Q.A. 

24. 

BUSINESS  DUAL.  MANAGEMENT 

PROCESS  Q.A. 

1.0  HR. 

25. 

QUALITY  CONTROL  MANAGEMENT 

PROCESS  Q.A. 

1.0  HR. 

26. 

ENGR.  TECHNOLOGY  OF  QUAL.  CONTROL 

P.A.  PROGRAMS 

2.0  HR. 

27. 

STATISTICAL  TECHNOLOGY  OF  QUAL.  CONTROL 

SPECIAL  PRODUCTS 

3.0  HRS. 

2B. 

applying  total  quality  CONTROL  AT  H.l. 

PROCESS  Q.A. 

1.0  HR. 

29. 

QUALITY  EDUCATION  1 TRAINING 

PROCESS  Q.A. 

1.0  HR. 

• Operating  Personnel 

• Basic  traininn  programs 

• Central  training  school  for 
soldering,  welding,  plastics 


• Production  operators/ 
quality  inspectors 

• Recertification  requirements 

• Time  lapse,  performance, 
job  assignment 

• Program  unique  training 

• Vestibule  and  OJT 

• Special  processes/unique 
criteria 

• Certi fication/recerti fi cat  ion 

• Monitored  via  computer  program 

• Product  Logistics  Support  Personnel 

• College  level  education 

• Product  technology,  logistics, 

program  courses 

• Training  services  course 
catalog 

• Correspondence  courses  for 
field  personnel 

• Available  to  other  personnel 

• Program  familiarization 

• Major  program  site  managers 

• Program  orientations 

• All  Personnel 

• Tuition  aid 

• In-house  courses 

• Vestibule  and  OJT 

• Special/New  Processes 

• Examples  - plated  wire  memory, 

tunnel  build,  heat  treat,  plating, 
non-destructive  inspection,  etc. 

• Specific  training  courses  in  place 

• Certification/periodic  recertifi- 
cation required 

• Time  lapse,  performance,  job 
assignment 
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Records 

• Individual  accounting  of  cert, 
status 

• Maintained/verified  by  product 
assurance  and  supervision 

• Effectiveness  measurement 
Effectiveness  Measurements 

• Computerized  quality  measurement 
system  (QMS-200) 

• Yield  and  defects  per  unit 

• Work  centers 

• Project 

• Major  problem  highlights 

• Daily  exception  reports 

t Operator  performance  report 

• Process  reports 

• Summaries 

• Management  level 

• By  project 

• Defect  sumnary 

• Interrogation  capability 

• By  part  number 

• By  assembly  number 

• By  problem  definition 
(cause  code) 

• By  operator 

• By  program 

• By  process 

• Computerized  cost  reporting 

• Daily  input/output 

• By  project  charge  number 

• By  functional  area  (supervisor) 

• Periodic  summaries 

• Periodic  management  reviews 


Motivation 

• Division-wide  program 

• Strive  toward  error-free 
performance  (step) 

• Individual  recognition  and 
monetary  awards 

• Management  support  and 
participation 

t Quality  awareness  program 

• Group  incentive  program 

• Group  recognition  and 
monetary  awards 

• Area/program  oriented 

• Joint  quality  production 

• Division  implementation 

• Program  recognition 

• Periodic  briefing  of  personnel 

• Success  recognition 

• Customer  provided  films  and 

lectures 

• Astronaut  visits 

• Launch  visits/tours 

t Periodic  roundtable  discussions 

• Director  level  quarterly 

(to  all  personnel) 

• Management  staffs  weekly 

• Operating  personnel  monthly 

Summary 

• All  personnel  trained/certified/ 
recertified 

• Division-wide  motivation  programs 

• Communications  roundtables 

• Detailed  records  maintained  and 
utilized 

• Continuous  effectivity  measurement 

• QMS  200 

• Cost  reports 

• Training/cert  records 
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• Diversity  in  government-imposed 
requi  rements 

• Variances  between  government 
agencies 

• Examples 

• Space  shuttle  - 30  day 
recert 

• Navy  programs  - 6 months 
recert 

• Special  requirements  for 
standard  processes 

• Variances  within  government  agencies 

• SAMSO  - parochial  critiques 
based  on  individual  PCO 
perceptions/ interpretations 

• Impact  contractor  costs 

• Questionable  contribution  to  mission 

effectiveness 


The  Product  Assurance  Training  Program  at 
The  Aerospace  Corporation 

Allan  J.  Boardman 

Key  Words:  Training,  Product  Assurance 

In  mid- 1976,  as  part  of  an  overall  intro-  perhaps  one  or  two  hours  per  week  for  ten  weeks, 

spective  examination  of  potential  improvements  For  these  offerings,  advanced  registration  and 

we  at  the  Aerospace  Corporation  might  make  in  management  approval  of  attendance  was  deemed 

fulfilling  our  role  in  mission  assurance,  a simple  necessary.  The  program  was  set  up  to  be  admin- 

but  relatively  novel  educational  concept  was  born.  istered  by  the  Corporate  Staff  Development  Office 

The  idea  was  to  present  an  in-house  on-going  but  with  the  technical  content  controlled  by  the 

series  of  lectures,  workshops,  demonstrations,  technical  staff.  All,  or  nearly  all,  of  the  sessions 

laboratory  «»ssions,  etc,  open  to  Aerospace  and  are  scheduled  during  working  hours  and  for  those 

SAMSO  alike  and  aimed  at  a broad-based  upgrade  who  are  forced  to  miss  a session  we  video  tape 
of  skills  and  sensitivities  in  the  product  assurance  and  record  as  appropriate. 

disciplines.  In  this  paper  I will  briefly  describe  Before  talking  more  about  the  nature  of  the 

the  nature  of  the  concept  which  has  developed  into  program,  let  me  indicate  the  kind  of  material 
the  present  product  assurance  training  program,  offered.  As  you  can  see,  in  the  attached  table, 
and  comment  on  its  effectiveness.  the  topics  are  very  diverse.  They  deal  not  only 

As  I am  sure  most  of  you  are  well  aware,  with  specific  technical  matters  but  with  virtually 
the  Aerospace  Corporation  serves  SAMSO  as  the  any  subject  which  could  conceivably  improve  the 
principal  source  of  General  Systems  Engineering  effectiveness  of  our  people  in  dealing  with  associ- 
and  Integration  (GSE&I)  for  a large  number  of  Air  ate  contractors  or  the  customer  or  otherwise 
Force  space  and  missile  programs.  Our  company  impact  mission  success.  To  illustrate  the  breadth 
has  a technical  staff  of  some  1700  engineers  and  of  the  program,  we  felt  that  a more  intimate 

scientists  specializing  in  the  various  disciplines  understanding  of  the  on-site  functions  of  govern- 

necessary  to  perform  this  GSE&I  role.  A large  ment  plant  representatives  would  be  useful  to 

fraction  of  the  staff  performs  a program  office  many  of  our  program  and  support  engineers,  so 

function  working  closely  with  our  customer  in  the  we  invited  a DCAS  representative  in  for  an  orien- 

technical  management  of  each  project.  Another  tation  talk.  Likewise,  the  head  of  a local  AFPRO 

large  group  is  organized  by  technical  discipline  group  resident  at  a nearby  associate  came  to  speak 

with  the  engineers  providing  specialty  services  to  with  us.  We  try  to  address  current  problems  and 
each  of  the  programs  as  needed.  These  two  future  technology,  materials  questions,  testing, 

groups  operate  as  a conventional  matrix  organiza-  audits  and  inspection,  future  trends,  etc.  We  are 

tion.  tapping  the  expertise  of  our  own  staff  and  that  of 

When,  two  years  ago,  we  set  about  looking  our  associate  contractors  and  vendors  who  have 
for  methods  of  self-improvement  several  thoughts  been  very  cooperative  in  providing  us  speakers  and 
arose.  Wouldn't  it  be  nice  if,  for  instance,  it  material. 

were  somehow  possible  to  take  some  deliberate  Since  the  PATP  material  has  direct  rela- 

steps  to  share  certain  "smarts"  among  engineers  tionships  with  both  our  overall  program  support 

with  the  objective  of  making  more  of  our  people  and  identified  problems,  it  has  been  possible  to 

aware  of  some  key  issues  which  affect  mission  charge  attendance  at  our  sessions  directly;  that  is, 

success  probability?  Can  a control  systems  against  program  funds.  Therefore,  the  separable 

expert  be  trained  to  recognize  the  inappropriate  costs  of  conducting  the  PATP  have  been  minimal, 

use  of  hydraulic  fittings,  or  an  optical  analyst  To  date,  there  have  been  well  over  6,000  hours  of 

sensitized  to  notice  shortcomings  in  some  parts  attendance  with  perhaps  500  persons  present  at 

handling  procedure?  Also,  wouldn't  it  be  nice  if  one  or  more  sessions. 

it  were  possible  to  get  more  mileage  out  of  certain  In  contrast  to  conventional  formal  educa- 

specialists  - - take  them  out  of  the  one-on-one  tion,  the  program  is  intended  to  be  extremely 

firefighting  mode  and  put  them  in  a position,  at  flexible  and  responsive  to  the  needs  of  our  staff, 

least  part-time,  where  they  could  train  or  educate  A particular  session  or  series  of  sessions  can  be 
many  people  at  once?  These  and  related  ideas  organized  and  held  at  the  request  of  any  group  with 

germinated  and  became  the  embryo  of  the  product  legitimate  need.  Sessions  can  be  easily  repeated 

assurance  training  program.  The  concept  took  or  tailored,  schedules  can  be  adjusted  as  required, 

shape  in  the  summer  of  1976  and  in  September  of  many  of  the  short  sessions  are  held  with  minimum 

that  year  we  announced  the  start  of  the  PATP.  notice  so  that  we  can  take  advantage  of  specialists 

Initially,  two  types  of  sessions  were  visiting  the  Los  Angeles  area  for  other  reasons, 

offered:  short  sessions,  one  or  two  hours  in  Management  is  encouraged  to  attend  and 

length,  were  held  on  a variety  of  relevant  topics.  promote  the  program  within  their  organizations. 

These  were  open  to  all  members  of  the  Aerospace/  Since  our  objective  is  not  to  make  everyone  a 

SAMSO  community  with  no  registration  or  manage-  specialist  in  every  topic,  but  rather  increase 
ment  approval  required.  Some  topics  could  not  awareness  of  key  issues,  it  is  appropriate  that 

be  well  presented  on  such  a scale  and  were  there-  personnel  at  all  levels  in  the  Aerospace/SAMSO 

fore  offered  as  extended  sessions,  running  complex  periodically  attend. 


Industry  Ideas  on 
Mission  Assurance 


Although  it  is  always  difficult  to  get  an 
exact  measure  of  the  efficacy  of  educational 
programs,  we  have  attempted  by  the  use  of  ques- 
tionnaires and  monitoring  of  the  sessions  to 
determine  the  strengths  and  the  improvable 
characteristics  of  the  PATP.  Responses  on  the 
questionnaires  have  been  overwhelmingly  positive 
and  the  vigor  of  discussion  at  most  sessions  also 
indicates  a high  level  of  interest  in  and  utilization 
of  the  material  being  presented.  These  responses 
suggest  a profitable  return  in  terms  of  improved 
personnel  effectiveness. 

We  recognized  early-on  the  impracticality 
of  adding  enough  manpower  to  our  central  product 
effectiveness  support  staff  to  cover  all  facets  of 
each  program.  Our  objective  with  the  PATP  is 
to  make  each  manager  and  engineer  who  deals  s 
with  program  elements  a bit  of  a product  assurance 
engineer  himself- -at  least  enough  of  one  to  spot  a 
problem  in  the  making  and  then  call  in  a specialist. 
Initial  results  suggest  the  concept  is  working. 

Mailing  address: 

Mr.  Allan  J.  Boardman 
The  Aerospace  Corporation 
2350  East  El  Segundo  Blvd. 

El  Segundo,  California  90245 


Thick  Fi  lm 
Technology  at  DuPont 


Hybrid  Memory  for 
Voyager  JPL) 

Hybrid  Microelectronics 
Survey  for  Space  and 
Missile  Use 

Space  Shuttle  Users 
Symposium  (Nationwide 
Teleconference) 

Defense  Documentation 
Center 

Spacecraft  Hydraulic 
and  Pneumatic  Integrity 

RTV  Silicones 

DCAS  Organization  and 
Functions 

Military  Specs  & Standards 


Allan  J.  Boardman  is  the  Director  of  the  Sensors 
and  Product  Effectiveness  Subdivision  within  the 
Electronics  and  Optics  Division  of  The  Aerospace 
Corporation.  Within  his  subdivision  are  the 
Reliability  and  Statistics  Department  and  the 
Components  and  Product  Engineering  Department, 
both  of  which  provide  engineering  support  services 
to  all  principal  SAMSO  programs.  He  has  been 
with  Aerospace  since  1962.  Prior  to  joining 
Aerospace  Boardman  was  Assistant  to  the  Vice 
President  at  Pickett,  1956-1959:  Staff  Project 
Engineer  at  Ramo- Wooldridge  Corporation, 
1959-1961.  He  received  his  bachelor's  degree  in 
aeronautical  engineering  from  the  Massachusetts 
Institute  of  Technology  in  1955  and  has  taken 
graduate  studies  at  University  of  Southern 
California  and  University  of  California  at  Los 
Angeles.  He  is  a member  of  the  Optical  Society 
of  America,  and  the  American  Institute  of 
Aeronautics  and  Astronautics. 


VLSI  Technology 

Integrating  Engineering 
Disciplines 

High  Reliability  Semi  - 
conductors  (Motorola) 

Microcircuit  and  Semi- 
conductor Packaging 

Sneak  Circuit  Analysis 

I.  Additional  Topics  to  be  Covered  as  Part  of 
the  Product  Assurance  Training  Program: 

1.  Inspection  of  Soldered  Connections 
and  Proper  Soldering  Techniques 
(Joyce  Gardner) 

2.  Use  of  the  Microscope  for  Quality 
Inspections  (Jim  Richardson) 

3.  Secondary  Breakdown  of  Semiconductors 
(Mel  Cohen) 

4.  Hybrid  Microcircuit  Packaging,  Applica- 
tion and  Testing  (Don  Fresh) 

5.  Program  Application  and  Use  of  MIL-STD- 
1546(19)  (Standardization  and  Control 
Program  for  the  Selection,  Application, 
Procurement  and  Management  of  Parts, 
Materials  and  Processes  for  Space, 

Launch  and  Re-Entry  Systems) 

6.  Program  Applications  and  Use  of  MIL-STD- 
1547(19)  (Technical  Requirements  for 
Parts,  Materials  and  Processes  for  Space, 
Launch  and  Re-Entry  Systems) 

7.  Benefits  of  Coordinated  Procurement  and 
Line  Surveillance 


8.  Use  of  LSI  in  Space,  Launch  and  Re-Entry 
System 

9.  Dormancy  Effects  on  Electronics  for 
Space  Systems 

NOTE:  Numbers  4,5,6  and  8 are  currently 
planned  for  extended  sessions. 

II.  Recommend  that  contractors  submit  their  ideas 
for  subjects  to  be  presented  under  the 
Aerospace/SAMSO  Product  Assurance  Training 
Courses  Program.  The  contractor  can  base  his 
suggestions  on  what  he  could  provide  (speaker) 
or  what  he  thinks  would  be  of  interest  based 
on  his  experience. 
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Experience  sharing.  ...  that  is,  informa- 
tion exchange.. . is  a key  element  in  any  mission 
assurance  endeavor.  This  has  been  recognized 
by  the  Industry  and  SAMSO  executives  who  set  up 
this  conference  and  it  has  been  emphasized  in 
each  of  the  keynote  addresses  of  the  conference. 
Indeed,  there  is  a considerable  degree  of  recog- 
nition among  all  of  us  in  this  business  that  there 
are  various  ways  in  which  experience  sharing  can 
be  accomplished  and  work  to  benefit  our  pro- 
grams. These  experience  sharing  methods  range 
in  scope  and  complexity  from  informal  conversa- 
tions between  two  individuals  to  highly  organized 
conferences  such  as  this  and  elaborate  formal 
information  exchange  systems  such  as  the  Defense 
Documentation  Center  (DDC)  and  the  Government 
Industry  Data  Exchange  Program  (GIDEP). 

Ail  forms  of  experience  sharing  have  value 
and  tend  to  aid  the  mission  assurance  effort. 
That's  the  overall  message  we  want  to  convey  in 
this  portion  of  our  Workshop.  Because  the  DDC 
and  GIDEP  types  of  experience  sharing  systems 
are  generally  familiar  to  most  of  us,  the  limited 
time  available  for  this  presentation  will  be 
devoted  to  a specific  application  of  experience 
sharing  to  the  prupose  of  enhancing  mission 
success  for  SAMSO  programs.  We  call  this 
effort  the  SAMSO/Aerospace  Experience  Sharing 
Program. 

In  the  next  few  minutes,  with  the  aid  of  a 
few  graphic  aids,  I will  acquaint  you  with  this 
program,  show  its  relationship  to  certain  facets 
of  GIDEP  and  seek  to  stimulate  thought  in  your 
minds  which  will  lead  to  fruitful  discussion  later 
today  in  the  Workshop  session. 

Objectives  of  the  SAMSO/Aerospace 
Experience  Sharing  Program  are  stated  in  Chart 
2. 

The  principal  functions  involved  in  imple- 
menting the  program  and  organizational  responsi- 
bilities for  their  performance  are  shown  in 
Chart  3.  The  planning  and  basic  administration 
of  the  program  are  carried  out  by  a small  section 
within  The  Aerospace  Corporation  known  as  the 
Reliability  and  Components  Data  Center  (RCDC). 

The  RCDC  maintains  four  distinct  data 
banks  (see  Chart  4).  Two  of  these,  the  SAMSO/ 
Aerospace  Experience  Retention  Data  Bank 
(ERDB)  and  the  SAMSO/Aerospace  Parts, 
Materials  and  Processes  Usage  Data  Bank  (PMP 
UDB)  are  unique  to  the  SAMSO/Aerospace  Exper- 
ience Sharing  Program. 

The  ERDB  is  a composite,  computerized 
bank  of  data  which  integrates  information  genera- 
ted in  SAMSO  programs  with  information  obtained 
from  various  sources  external  to  SAMSO  pro- 
grams. Additional  details  of  the  composition  of 
this  data  bank  will  appear  in  the  next  chart. 

The  PMP  UDB  is  a computerized  bank  of 
equipment  parts,  materials  and  processes  lists 


for  specific  SAMSO  programs.  Having  these  PMP 
lists  in  the  computer  permits  a rapid  identification 
of  programs  which  may  be  impacted  by  specific 
GIDEP  Alerts  or  other  conditions  of  immediate 
concern  revealed  by  any  of  the  various  experience 
sharing  media. 

The  ERDB  and  UDB  are  augmented  by  two 
of  the  microfilm  data  banks  made  available  by 
GIDEP  to  Aerospace  as  one  of  its  participating 
organizations.  These  are  the  GIDEP  Engineering 
Data  Bank  (EDB)  and  the  GIDEP  Reliability  and 
Maintainability  Data  Bank  (R/MDB). 

The  left-hand  side  of  Chart  5 shows  the 
various  types  of  documents  which  make  up  the 
SAMSO/Aerospace  Experience  Retention  Data 
Bank.  The  right-hand  side  of  this  chart  delineates 
the  major  types  of  routine  and  special  outputs 
distributed  by  the  RCDC,  utilizing  information 
retrieved  from  the  data  banks. 

Charts  6.  1 through  6.  4 show  the  flow  of 
the  various  types  of  information  in  the  SAMSO/ 
Aerospace /Industry  experience  sharing  network. 

Chart  6.  1 shows  the  flow  of  information 
from  SAMSO  programs  into  the  Aerospace  Relia- 
bility and  Components  Data  Center.  Some  of  this 
data  is  received  by  the  RCDC  direct  (as  a con- 
tract C.DRL  addressee),  but  at  present  most  of  it 
is  received  indirectly  via  the  cognizant  SAMSO 
and/or  Aerospace  program  office. 

Chart  6.2  shows  the  flow  of  GIDEP  Failure 
Experience  Data  Bank  information  (Alerts).  Note 
here  that  the  Aerospace  RCDC  serves  as  a central 
redistribution  point  of  GIDEP  Alerts  to  all  Aero- 
space Corporation  and  SAMSO  offices.  In  the 
case  of  SAMSO  distribution,  GIDEP  Alerts  are 
addressed  by  the  RCDC  to  the  Air  Force  Discrep- 
ant Parts  and  Components  Control  Program 
(DPCCP)  monitors  for  each  of  the  various  System 
Program  Offices  (SPOs),  as  identified  in  a list 
supplied  by  the  Directorate  of  Acquisition  Support, 
Systems  Effectiveness  Branch  (SAMSO/AWS). 

Chart  6.3  shows  the  flow  of  GIDEP 
Engineering  Data  Bank  information.  Here  again 
the  Aerospace  RCDC  serves  as  the  central  point 
of  receipt,  repository  and  redistribution  for 
Aerospace  and  SAMSO  offices. 

Chart  6.  4 shows  the  flow  of  information 
from  sources  other  than  SAMSO  programs  and 
GIDEP  into  the  Aerospace  RCDC.  Some  of  this 
information  is  received  direct  by  the  RCDC  and 
some  of  it  indirectly  via  SAMSO  or  other  Aero- 
space offices. 

The  flow  of  output  from  the  Aerospace 
RCDC  is  shown  in  Chart  6.  5.  Here  it  is  seen  that, 
at  the  present  time,  all  distribution  of  RCDC- 
originated  information  to  SAMSO  program  contrac- 
tors is  channeled  through  Aerospace  and/or 
SAMSO  offices.  The  feasibility  and  ramifications 
of  direct  distribution  from  the  RCDC  to  contractors 
remains  to  be  studied. 
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In  Chart  7 we  have  cited  a few  of  the 
problems  that  appear  to  confront  us  (in  varying 
degrees  of  seriousness)  in  administering  and 
effectively  utilizing  the  SAMSO/ Aerospace 
Experience  Sharing  Program.  Although  there 
is  insufficient  time  available  to  discuss  any  of 
these  in  detail  in  the  presentation,  it  is  our  hope 
that  a meaningful  exchange  of  ideas  on  at  least 
some  of  them  will  take  place  in  the  workshop 
session  which  follows. 


Program  Objectives 


• Acquire  and  permanently  preserve 
technical  and  managerial  experience 
information  from  SAMSO  programs  and 
available  outside  sources. 

• Integrate  SAMSO  experience  data  with 
similar  data  from  outside  sources  into  a 
central,  rapid-access  data  bank. 
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• Provide  data  retrieval  and  dissemination 
services  which  effectively  facilitate  and 
encourage  application  of  past  experience 
information  to  enhance  mission  assurance 
for  current  and  future  SAMSO  programs. 
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Problem  Areas 

• Product/Process  Identification 

Multiple  P/Ns,  nomenclature, 
variations  of  basic  type 

• Information  Integrity 

Accuracy,  objectivity,  completeness, 
timel iness 

t Information  Significance  Appraisal 

Criticality  rating,  scope  of  impact, 

usefulness 

t Closing  the  Loop 

Action  definition,  assignment, 
motivation,  follow-up,  data  bank 
up-date/purge 

§ Distribution  Lists 

Establishing,  maintaining. 
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Experience  Sharing 


INDUSTRY'S  APPROACH  TO  PERSONNEL  MOTIVATION 
AND  TRAINING  AND  SAMSO/AEROSPACE/ 
INDUSTRY  EXPERIENCE  SHARING 
DISCUSSION  GROUP  SUMMARIES 

I.  QC  Circles  and  Related  Programs 

• QC  circles  have  been  tried  with 
some  successes 

• Raise  job  satisfaction, 
morale,  productivity,  etc. 

• Techniques  relatively  new  in 
American  industry 

therefore: 

• Expose  other  organizations  in  the 
SAMSO  community  to  techniques, 
successes,  problems,  benefits 


• Common  perception  of  the  benefits 
of  a good  experience  sharing  system 

• Simply  not  enough  Interest  anywhere 
in  systematic  Improvement  in  experi- 
ence sharing;  not  set  up  organiza- 
tionally to  do  a better  job 

• List  of  deterrents  extensive  - 

• Fear  of  embarrassment 

• Too  much  trouble  to  input 

• Proprietary  info 

• Legal  questions 

• etc. 

therefore: 


• Publish  experiences,  guide- 
lines, case  studies 

• Hold  seminars  for  inter-company 
sharing  at  working  level 

ACTION:  Industry  - individual  initiative 

• Organizational  - specialty 
groups 

II.  In-Plant  Motivators  - Use  of  Techniques 
from  the  Behavioral  Sciences 

• Many  managers  from  "old  school” 
uninformed  about  effective 
techniques  or  reluctant  to  use  them 

• Studies  indicate  needs  - e.g., 
grossly  differinq  perceptions  of 
people,  job,  goals  as  function  of 
organizational  level 

therefore: 

• Provide  greater  exposure  of  methods 
to  management 

• Train  customer  personnel  on  Impact 
of  interactions  with  contractor 
(example:  gen.  nuts  and  bolts) 

ACTION:  Industry,  government,  professional 

societies 


• Must  examine  list  of  deterrents, 
overcome  or  override  with  con- 
tractual requirements  and/or 
incenti ves 

ACTION:  SAMSO/AEROSPACE  Task  Force  to 

review  and  propose  specific 
recommendations 

1 1 1 B . Training 

• No  major  industry  problems  identi- 
fied in  "operator”  training 

• Challenge  to  Industry  to  apply 

modern  technology  (computer  aids, 
e.g.)  to  streamline  traininq 

• Government  and  Industry  both  need 
some  form  of  training  programs  to 
pass  along  ipertise  in  RFP  and 
proposal  preparation,  requirements 
translations,  etc. 

IV.  Customer  Involvement  and  Organizational 

Influences  on  Motivation 

• Little  SAMSO  recognition  of  value 

of  motivational  programs 

• Motivational  programs  seldom 

directly  funded 

therefore: 


Also  consider: 

• Motivation  is  not  something  we  "do 
to  people"  - rather  create  the  right 
environment  to  enable  generators  to 
operate 

• Minimize  the  extent  to  which  check- 
ing other  people's  work  Is 
necessary  - operator  accountability 

• Develop  self-sustaining  techniques 


• RFP's  should  encourage  motivational 
programs  as  a line  item  where  ROI 
can  be  demonstrated 

• SAMSO  should  employ  motivational 
programs  in  their  own  organiza- 
tions - lead  the  way 


• SAMSO  should  encourage  contractors 
to  recognize  their  outstanding 
performers/groups 


• SAMSO  should  prepare  and  distribute 
list  of  lessons  learned 
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ACTION:  SAMSO 


Also  consider: 

• SAMSO  publicity  of  successful 
contractor  efforts  with  care 


• SAMSO/industry  working  group  and/or 
bulletin  to  investigate  and  serve 
motivation 

Training 

Experience  sharing 

Highlights  from  Paper  Presentations  Personnel 
'Motivation,  Training  and  Experience  Sharing 


Motivation 


• Motivation  comes  from  within 

• Put  time  limit  on  program  or  modify 
periodically 

• Use  formal  organization 

• Major  benefits  from  QC  circles,  peps, 

team 

• Supervisory  training 

• Teaching  people  how  to  define 
problems  and  develop  solutions 

• Communications 

• Job  enrichment 

• Between  4:1  to  9:1  cost-benefit 

ratio 

• Involve  all  levels  of  management 

• Suggestion  systems  versus  employee  meetings 

• Consider  more  group  incentive  programs 

• Restructure  jobs  to  include  planning, 
scheduling,  controlling,  innovating 

• Certify  manufacturers  to  perform  own 
inspection 


Training 


• Technical  personnel  inter-disciplinary 
training  accomplished  by  short  on-the- 
job  courses.  Related  to  work. 

• Variable  government  certification 
programs 

Experience  Sharing 


• Large  payoff  from  technical  and 
management  data  banks 


- 
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Closing  Address  by:  Albert  D.  Wheelon 


After  three  days  on  the  subject  of  mission 
assurance,  I expect  you  have  heard  just  about  every- 
thing that  can  be  said  on  the  subject.  It  is  not 
easy  for  me  to  say  something  new.  I will  comment 
on  a few  of  the  more  important  items  raised  in  this 
conference,  and  will  add  a few  perspectives  of  my 
own. 

First,  let  me  say  that  industry  is  in  gocu  shape 
today.  It  can  be  better,  but  it  has  been  lots 
worse.  I recall  ballistic  missiles  in  1957  and 
1958  - 20  years  ago,  when  every  attempt  to  launch 
was  an  adventure,  and  most  of  our  products  ended  up 
in  the  water.  I remember  the  start  of  Intelsat  IV 
10  years  ago.  At  that  time,  S30M  of  the  S100M 
revenue  was  riding  on  orbital  performance.  We 
needed  to  get  5 years  successful  performance  in 
orbit  to  cover  our  costs.  The  first  Intelsat  IV 
bird  is  now  7.3  years  old.  Today,  we  are  all  build- 
ing and  performing  to  higher  standards.  This  change 
is  the  result  of  the  efforts  of  people  in  this  room 
who  have  made  steady  progress  each  year. 

The  question  is:  How  do  we  progress  from  here? 

That  was  the  purpose  of  this  conference.  I 
remarked  to  Tom  Morgan  a year  ago  that  such  meet- 
ings - held  under  Air  Force  auspices  - are  difficult 
for  industry  to  hold  by  itself  in  view  of  antitrust 
laws.  This  conference  is  valuable  and  should  be 
repeated  regularly.  But  it  is  not  an  end  in  itself. 

I would  like  to  ask:  How  many  people  in  this  room 
will  do  "hands-on"  labor  on  space  hardware  this 
year?  I will  not.  How  many  will  design  and  make 
drawings  for  space  hardware  this  year?  I will  not. 
The  answer  to  both  questions  is  "Not  too  many." 

The  fact  is  that  the  destiny  of  spacecraft  is  in 
the  hands  of  people  outside  this  room. 

We  in  the  room  are  like  the  clergy  in  the  early 
Christian  churches.  We  can  teach;  we  can  exhort; 
we  can  encourage  - but  we  cannot  do  the  work  of  the 
world.  Only  by  enlightened  evangelism  can  we  excite 
those  who  will  do  it  - to  do  so  superbly  well.  If, 
like  the  medieval  clergy,  we  occupy  ourselves  with 
the  refinements  of  dogma,  we  will  have  failed  in 
our  primary  purpose. 

How  do  we  reward  and  encourage  people  for  working 
hard  to  provide  Mission  Assurance  - real  spacecraft 
operating  flawlessly  in  orbit?  I believe  the  answer 
is  "Not  very  well."  We  have  the  stick  but  not  the 
carrot. 

A common  misperception  by  Government  is  that  indus- 
try is  motivated  solely  by  profit.  Certainly,  the 
Individual  engineer  who  directly  affects  mission 
assurance  Is  far  removed  from  the  Influence  of 
profit.  So  it  is  the  management  we  are  talking 
about.  The  view  of  modern  management  is  far 
broader  than  corporate  profits.  Continuity  and 
dignity  of  employment  are  high  on  the  list  of  cor- 
porate goals.  That  is  why  George  Meany  is  opposed 
to  licensing  U.S.  products  to  foreign  firms  who 
then  destroy  the  firms  and  the  jobs  that  created 
those  products  by  predatory  pricing.  The  profits 
that  come  from  those  licensing  arrangements  in  no 


way  compensate  people  for  the  suffering  of 
unemployment  created  thereby.  As  a further  example, 
1916  and  1977  were  the  most  profitable  years  in  the 
history  of  my  part  of  the  Hughes  Aircraft  Company. 
But  for  me  they  were  the  most  painful  because  we 
had  to  lay  off  1000  of  our  5000  people.  I believe 
that  the  Government  does  not  yet  recognize  the 
broader  motivation  of  modern  industrial  management. 
When  it  does,  it  will  recognize  that  the  present 
incentives  for  mission  assurance  are  at  least 
inadequate  and  sometimes  counterproductive. 

Well,  what  does  counterproductive  mean?  It  means 
that  if  we  build  a 10  year  spacecraft  rather  than 
a 5 year  spacecraft,  we  will  get  exactly  half  as 
many  orders.  That  means  half  as  many  jobs.  The 
cruel  consequence  of  maintaining  operating  systems 
in  space  is  that  a short  orbital  life  means  more 
sales  — and  hence  more  jobs.  You  say  that  will 
eventually  catch  up  with  the  less  reliable  supplier 
But  does  it?  Do  we  have  a mechanism,  sanctioned  by 
ASPR,  which  allows  the  government  to  recognize  the 
superior  performance  of  one  spacecraft  supplier  vs. 
another?  Can  the  government  refer  in  any  meaningful 
way  to  prior  performance  in  orbit  when  making  source 
selection  decisions?  You  and  I do  this  in  buying  a 
TV  set  or  a car.  But  I submit  that  the  Government 
has  not  yet  found  a way  to  recognize  performance; 
in  contrast  to  liturgical  discipline  of  formal 
product  assurance  plans. 

We  all  see  the  coming  of  Shuttle  as  a new  day  in 
space,  providing  new  directions  for  design  and  test. 
But  the  other  side  of  the  coin  is  the  wind-down  of 
Thor  Delta,  Atlas  Centaur,  and  Titan.  How  do  we 
motivate  those  people  who  will  soon  be  looking  for 
new  jobs,  especially  since  we  still  have  a lot  of 
satellites  to  fly  with  those  rockets? 

This  conference  has  discussed  the  desirability  of 
focusing  space  business  on  a few  electronic  com- 
ponents that  will  ensure  the  reliability  we  need. 

I believe  this  is  important  especially  in  view  of 
the  terrific  diversion  of  integrated  circuit  manu- 
facturers to  the  commercial  market.  But  such 
focusing  has  already  taken  place  in  ways  that  many 
of  us  do  not  recognize. 

For  instance,  I believe  that  everyone  in  the  indus- 
try buys  titanium  tubing  from  a single  supplier;  we 
are  critically  dependent  on  his  quality  standards, 
which  have  their  ups  and  downs.  I believe  that 
everyone  buys  nickel-cadmium  batteries  from  one 
supplier,  whose  primary  emphasis  is  building  batter- 
ies for  golf  carts  and  fork  lift  trucks.  It  is  no 
secret  that  some  battery  lots  are  better  than 
others. 

Consequently,  I am  making  two  main  points:  The 
first  is  that  there  is  a surprising  amount  of 
focusing  - the  pyramid  standing  on  its  tip  — and 
that  we  all  depend  on  the  performance  of  a few 
people,  who  have  periodic  shortfalls.  The  second 
is  that  the  problem  is  broader  than  electronic 
parts.  We  judge  tha  . half  our  orbital  problems  are 
mechanical  and  half  are  electronic.  We  must  keep  a 
balanced  view  of  our  liabilities. 

We  are  trying  to  enhance  Mission  Assurance  within 
fixed  economic  boundaries.  I like  very  much 
General  Low's  term  "Affordable  mission  assurance." 
That  is  the  reality  of  our  situation.  Whether  we 
be  on  firm  fixed  price  or  cost  pins  contract,  there 
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is  a limit  to  the  amount  of  money  available  for 
Mission  Assurance.  And  it  seems  it  is  never 
enough.  Certainly  it  is  not  enough  to  nail  every 
question  down  firmly.  Whether  we  like  it  or  not, 
there  continues  to  be  an  element  of  chance  in  our 
business.  The  problem  is  to  know  the  odds  and 
weigh  them  in  our  favor.  And  to  know  how  much  is 
enough! 

Mission  Assurance  must  be  woven  into  the  fabric  of 
program  management.  It  cannot  afford  to  be  a thing 
apart.  We  employ  a system  at  Hughes  that  helps 
this  weaving.  We  devote  each  Friday  to  an  in-depth 
review  of  our  ten  major  spacecraft  programs.  I 
chair  that  meeting  and  our  Product  Assurance  people 
participate.  They  are  expected  to  provide  an  inde- 
pendent view  on  how  things  are  going  in  each  pro- 
gram. They  do  so.  It  is  a healthy  system  which 
seeks  to  surface  problems  and  uncertainties  while 
the  spacecraft  are  still  here  on  earth  and  we  can 
fix  them.  The  program  managers  have  come  to  regard 
this  as  a forum  in  which  they  can  get  management 
support  for  all  their  problems.  I believe  it  pro- 
vides a healthy  forum  for  integrating  mission 
assurance  into  the  lifeblood  of  our  activities.  I 
gather  that  other  companies  use  essentially  the 
same  technique. 

This  conference  is  the  first  attempt  to  examine  all 
the  influences  that  affect  Mission  Assurance.  It 
has  covered  design,  manufacturing,  quality  assur- 
ance, testing,  subcontractor  management,  software 
management,  contractual  incentives,  and  human  moti- 
vation. It  is  possibly  a turning  point  in  the  con- 
duct of  our  business.  It  is  important  that  the 
concept  developed  here  should  not  be  dropped  and 
that  effective  follow-up  be  maintained. 

We  owe  a considerable  debt  to  General  Morgan,  who 
was  the  principal  force  behind  this  meeting.  I 
would  like  to  add  my  best  wishes  to  those  of  others 
as  General  Morgan  leaves  SAMSO  and  a valuable 
legacy  to  us  all. 
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process  by  recording  the  non-text-book  aspects  of 
our  business. 


Comments  by  Mr.  T.A.  Meaker 

Head  of  Product  Assurance,  Telecommunication 
Satellites  Department  EUROPEAN  SPACE  AGENCY  - 
Netherlands 

Mr.  Chairman,  Ladies  and  Gentlemen: 

In  being  asked  to  give  a summary  overview  of  the 
conference  I feel  rather  like  a Freshman  being  asked 
to  comment  at  a Graduation  ceremony.  Nevertheless 
I would  like  to  make  a few  observations  and  perhaps 
I may  begin  by  making  a few  comments  about  the 
European  space  efforts. 

As  the  chairman  has  indicated  I work  for  the 
European  Space  Agency  which  is  roughly  the  European 
equivalent  of  NASA.  I am  therefore  essentially  a 
member  of  a Government's  Organization  i.e.  the 
customer,  and  interface  frequently  with  contractors 
and  sub-contractors  etc.  At  the  moment  the  main 
efforts  at  ESA  are  devoted  to  the  Telecommunication 
Satellite  Programmes,  Spacelab  for  Shuttle  Applica- 
tions and  Scientific  Satellites.  I am  a member  of 
the  Telecommunication  Satellites  Department  headed 
up  by  Dr.  R.  Collette  and  at  the  moment  we  are 
involved  in  Maritime,  Indirect  and  Direct  Television 
Broadcasting  and  Telecon  service  satellites. 

ESA  also  has  an  extensive  ground  based  communica- 
tions network  and  is  involved  in  a fair  amount  cf 
basic  research  activity. 

I would  like  to  mention  that  the  major  consortia  in 
Europe  have  worked  with  a number  of  American  space 
organizations  and  thus  you  can  share  in  the  suc- 
cesses we  have  had  in  our  space  programmes. 

I believe  the  Workshop  form  of  this  conference  has 
enabled  an  eyeball  to  eyeball  exchange  of  ideas, 
criticisms  and  suggestions  in  rather  critical  areas. 
The  structuring  of  the  workshops  has  clearly  been 
very  carefully  thought  out  and  the  control  of  the 
rather  complicated  committee  sessions  has  been 
extremely  wel 1 handled. 

A characteristic  of  the  conference  that  has  been 
apparent  to  myself  is  that  realistic,  pragmatic  dis- 
cussion has  taken  place.  There  has  been  some 
extremely  hard,  blunt  argument  and  controversies 
have  been  discussed,  without  animosity.  What  is 
probably  more  important,  with  the  many  service  mem- 
bers present,  is  that  there  has  been  no  "pulling  of 
rank"  to  win  a point  in  debate. 

A conference  of  this  description  is  rather  unique. 
The  conference  has  been  convened  by  Government  and 
a very  wide  selection  of  industrial  engineers  has 
attended.  With  such  representation,  on  both  sides 
as  it  were,  I think  a redeeming  feature  of  the 
conference  has  been  that  most  of  the  people  con- 
cerned have  ADMITTED  problems  and  inadequacies. 
Additionally,  limitations  have  been  recognized  and 
advice  has  been  accepted.  Amidst  all  this  ammuni- 
tion for  dissent,  humor  and  mutual  respect  have  been 
maintained.  I am  also  very  pleased  to  have  noted 
that  no  "sacred  cows"  have  been  acknowledged. 

It  seems  to  me  that  we  are  constantly  re-inventing 
the  wheel.  The  reason  for  this,  in  many  cases,  is 
that  there  is  inadequate  communication  and  too  much 
customer-contractor  class  distinction.  This  sort  of 
conference  will  hopefully  reduce  the  re-invention 
process  by  recording  the  non-text-book 


If  you  will  bear  with  me  for  a few  moments  I would 
like  to  make  a few  points  in  which  I am  particularly 
interested. 

We  are  constantly  and  continuously  being  penalized 
for  our  failure  to  quantitatively  define  the  mission 
early  in  programme.  Commensurate  with  this  comment 
I would  like  to  make  the  observation  that  although 
the  customer  invariably  makes  the  policy,  this 
policy,  whether  we  like  it  or  not,  is  only  defined 
by  the  specifications.  Now,  as  we  all  know,  speci- 
fications do  change  and  we  should  realize  that  with 
the  changes  in  the  specifications  so  our  policies 
are  also  changing.  In  many  cases  there  is  a very 
tenuous  connection  between  the  final  specifications 
and  the  policy  makers.  Perhaps  some  sort  of  mech- 
anism should  be  established  which,  from  time  to 
time,  analyses  the  specifications  to  determine 
precisely  what  the  changes  in  the  policy  are. 

I find  the  title  and  the  mixed  discipline  content, 
of  this  conference  very  satisfying.  In  the  space 
business  with  so  many  disciplines  and  skills 
DEPENDENTLY  involved,  it  is  clear  that  no  single 
type  of  expertise  is  more  important  than  another. 

Hence  the  el imination  of  the  second  class  citizen 
concept  for  product  assurance  engineers,  contract 
officers,  finance  people,  sheet  metal  workers  etc. 
is  fundamentally  important.  I believe  that  the 
design  engineer  is  not  a supreme,  isolated, 
sacrosanct  paragon  and  the  quicker  the  integrated 
concept  of  our  work  and  our  people  is  implemented 
the  more  successful  our  missions  will  become. 

We  are  living  in  an  age  of  inventiveness,  innovation 
and  initiative  with  the  predictable  result  of  exotic 
technologies  and  custom  designs.  As  mentioned  at 
this  conference  the  promulgation,  in  this  diverse 
environment,  of  AGREED  standardization  in  specifi- 
cations etc.  is  tremendously  important.  The  stan- 
dardization '..ill  to  a certain  extent  restrict  the 
use  of  new  products  but,  in  the  long  run,  I believe 
this  will  lead  to  higher  assurance  of  mission 
success. 

In  our  business  I am  becoming  increasingly  aware  of 
the  interface  problems  that  are  besetting  our 
programmes.  In  this  respect,  and  this  has  been 
emphasized  in  one  or  two  of  the  workshops,  the 
importance  of  CADM  and  technical  interface  manage- 
ment cannot  be  overestimated.  In  the  context  of 
this  remark  I would  like  to  mention  the  difficulties 
we  are  currently  having  in  defining  the  shuttle 
payload  interfaces  from  the  information  currently 
available. 

I would  like  to  say  a few  words  about  WORKMANSHIP. 
Workmanship  has  been  extensively  discussed  at  this 
conference,  both  in  absolute  terms  and  in  the 
transition  process  from  design  to  hardware  manufac- 
ture. A number  of  speakers  have  passed  comment 
about  the  number  of  PH.D's*  in  a particular  comnitte 
and  have  drawn  the  conclusion  that  the  committee 
result  would  therefore  be  a good  one  because  of  the 
"large  amount  of  brainpower  present."  I naturally 
recognize  the  importance  of  PH.D's  and  without  their 
very  specialist  research  knowledge  many  of  our 
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problems  would  remain  unsolved.  I do,  however 
think  that  there  is  a very  big  imbalance  between 
workmanship  skills/industrial  engineering  and  the 
PH.D.  Perhaps  we  should  recognize  a concept  of 
"PH.D  in  workmanship/experience."  The  implementa- 
tion of  this  concept  may  in  fact  be  more  effective 
in  reducing  the  final  10%  of  defects  we  are  cur- 
rently looking  for.  The  previous  speaker  made 
some  extremely  perceptive  comments  about  motivation 
and  continuation  of  hardware  standards;  I would 
like  to  say  that  I fully  endorse  his  observation. 

Perhaps  on  the  government/customer  side  and  bearing 
in  mind  the  difficult  financial/budget  climate,  I 
would  like  to  make  a brief  comment  on  payments. 

Some  speakers  have  said  "you  get  what  you  pay  for." 

I believe  we  should  also  remember  that  you  often 
"get  what  you  do  not  pay  for"  and  this  can  be  the 
"FAILURES."  This  comment,  of  course,  applies  not 
only  to  government/customer  but  also  to  main  con- 
tractors when  subcontracting  to  lower  tier 
contractors. 

As  a concluding  comment,  I believe  this  conference 
has  been  very  successful,  but  the  ultimate  success 
depends  on  the  continuing  close  relationship  and 
integration  of  government  (customer)  and  industry. 
Initiative,  as  shown  in  convening  this  conference, 
will  always  pay;  complacency,  from  now  on,  is  the 
enemy.  The  customer  and  the  contractor  must  recog- 
nize and  £se  each  others  contributions  and 
1 imitations. 

I would  like  to  congratulate  the  organizers  of  this 
conference  in  producing  such  a co-ordinated  and 
current  event.  The  conference  is  as  much  the  people 
who  attend  and  the  contributions  they  have  made 
have  been  invaluable.  I thank  you  for  the  oppor- 
tunity to  attend  and  would  welcome  you  to  visit  ESA 
if  you  are  in  Europe. 


Gerald  E.  Lutz 


If  there  are  any  questions,  I will  try  to  answer 
them.  If  not,  the  meeting  is  adjourned. 


Director  of  Quality  Assurance 
Fairchild  Space  and  Electronics  Company 


I have  been  most  impressed  with  the  vast  array  of 
recommendations  that  have  been  developed  and  pre- 
sented during  this  workshop.  Not  only  are  the  work- 
shop coordinators  to  be  complimented  on  a thorough 
summarization,  but  each  of  you  deserve  plaudits  for 
your  forthright  comments  and  suggestions. 

If  this  symposium  is  to  be  considered  a success, 
then  positive  and  constructive  change  must  be 
evolved  as  a result  of  the  many  comments  provided. 

It  is  our  intent  to  continue  to  support  SAMSO  in 
the  development  of  specific  actions  to  the  degree 
they  desire. 

Towards  that  objective  then,  papers  and  workshop 
summaries  will  be  published  to  serve  as  the  source 
material.  We  view  this  workshop  as  but  another 
step  in  a continuing  effort  to  enhance  Mission 
Assurance. 

The  ultimate  measure  of  success  will  be  space  per- 
formance in  the  future  that  meets  or  exceeds  mis- 
sion performance  requirements  to  a greater  degree 
than  in  the  past.  This  is  especially  challenging 
as  mission  durations  will  extend  and  mission  com- 
plexity is  increased.  That  this  can  be  done,  I 
have  no  doubts.  How  well  we  utilized  this  open 
forum  to  achieve  these  results  will  be  the  yard- 
stick of  our  success. 

I would  like  to  assure  SAMSO  and  Aerospace  that 
Industry  stands  ready  to  support  the  implementation 
of  the  resulting  recommendations  in  whatever  manner 
may  be  appropriate  - whether  this  be  special  task 
forces,  additional  workshops,  special  evaluations 
or  whatever  appears  warranted. 

Our  objectives  parallel  yours  - improve  Mission 
Assurance  and  improve  costs. 

It  would  take  the  rest  of  the  day  to  acknowledge 
the  many  people  whose  dedication  and  perseverance 
made  this  conference  successful,  however,  I will 
take  the  opportunity  to  reflect  on  them 
collectively: 

• Industry  Program  Committee 

• SAMSO/ Aerospace  Program  Committee 

• Ad  Hoc  Committee 

• Workshop  Co-Chairmen 

• Workshop  Co-ordinators 

• Discussion  Moderators 

• Speakers 

• Panelists 

• NSIA  Staff 

• Administrative  Personnel 

• Each  of  you  as  a contributor 

Last,  but  certainly  not  least,  those  who  assisted 
with  registration,  message  center,  and  worked  late 
Into  the  night  making  Vue-Graphs  and  supporting 
these  reviews.  Thank  you  again  for  making  this 
workshop  productive. 
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APPENDIX  B 

SUMMARY  OF  SAMSO/INDUSTRY  EXECUTIVE  SESSIONS 

22  June  1977 
7 July  1977 
20  July  1977 


Opening  Remarks 

The  items  discussed  at  the  meetings  were  suggested 
by  the  contractors.  Although  the  subjects  were 
similar  at  each  meeting,  the  emphasis  and  relative 
importance  placed  on  the  discussion  varied  among 
the  meetings.  In  opening  the  meetings,  General 
Morgan  pointed  out  that  while  the  record  of  space- 
craft success  is  very  good,  the  impact  of  a small 
number  of  space  failures  can  have  an  exceptionally 
adverse  effect.  This  impact  goes  beyond  the  initial 
operational  loss  and  the  staggering  loss  of 
resources  at  many  levels.  The  ability  to  obtain 
Congressional  support  for  future  budget  allocations 
is  seriously  impeded.  Management  at  all  levels 
must  recognize  that  for  space  applications,  we  must 
strive  to  achieve  perfection.  At  times  this  may  be 
initially  costly;  however,  the  alternative  of  fail- 
ure is  even  more  costly. 

Incentives 

Achieving  the  appropriate  balance  of  contract 
incentives  is  considered  very  important  in  motivat- 
ing contractor  management  in  the  right  direction. 
Several  examples  of  how  incentives  can  actually 
contribute  to  failures,  anomalies  and  performance 
degradation  were  given.  For  example,  schedule 
incentives  with  a fixed  price  contract,  when  tech- 
nical problems  are  encountered,  could  encourage 
short  cuts.  Unrealistic  estimates  of  cost  and 
schedule  often  result  in  decisions  have  an  adverse 
effect  on  mission  performance.  Timing  of  incentive 
payments  is  also  important  to  maximize  the  motiva- 
tion desired. 

Performance  oriented  incentives  significantly 
greater  than  cost  or  schedule  are  considered  a 
strong  tool  for  focusing  management  attention  on 
those  areas  that  enhance  the  probability  of  mis- 
sion success.  Properly  drawn,  incentives  could  be 
the  guide  for  every  management  decision  and  trade- 
off in  deciding  how  and  where  to  apply  resources. 
The  following  are  additional  ideas,  comments  and 
suggestions  concerning  incentives  discussed  at  the 
meeting. 

t Some  attendees  felt  that  multi-year  on- 
orbit  performance  incentives  that  are 
paid  downstream  are  not  effective.  The 
performance  of  contractor  personnel  is 
graded  in  the  present,  not  in  the  future. 

• Early  payment  of  incentives  when  initial 
performance  is  achieved  is  effective. 

We  might  consider  the  use  of  incentives 
in  the  front  end  of  the  program  and 
assess  penalties  if  full  performance  is 
not  attained. 

• Award  fees,  properly  structured  and 
operated,  are  a good  management  tool. 

Both  parties  should  have  commonality  of 
purpose  and  mutual  understanding  of  the 
award  fee  plan. 

• Contract  incentives  can  be  used  by 
management  to  motivate  personnel  to 
avoid  practices  that  might  inadvertently 
degrade  the  product.  Incentives  should 
be  passed  down  to  the  first  line  super- 
visors or  lower  in  "real  time." 


• Both  parties  should  have  a clear 

understanding  of  the  incentive  structure 
before  starting  work.  Rules  should  be 
published  in  advance.  Also,  there  should 
be  no  conflict  between  incentives,  i.e., 
cost  schedule  vs  performance. 

Motivation 

There  was  a general  feeling  that  one  of  the  most 
important  attributes  of  mission  assurance  is  motiva- 
tion of  the  work  force  and  instilling  an  awareness 
of  the  importance  and  criticality  of  spacecraft 
hardware.  Several  contractors  have  implemented 
formal  programs  for  this  purpose.  It  was  pointed 
out,  however,  that  formal  programs  often  lose  their 
vigor  after  a period  of  time  and  must  be  constantly 
updated.  It  was  felt  that  to  be  effective,  motiva- 
tional programs  must  have  complete  support  of  top 
management.  The  motivational  process  must  also 
"flow  down"  to  the  subcontractors  and  vendors. 
Techniques  described  included  cash  awards,  expense 
paid  trips  to  witness  launches,  movies,  participa- 
tion of  AF  personnel  and  presentations  on  how  the 
products  and  people  contribute  to  the  space  pro- 
gram. Many  felt  it  is  essential  to  instill  in  all 
employees  a personal  feeling  of  responsibility  and 
a feeling  of  involvement.  The  motivation  process 
also  involves  a sharing  of  successes  with  personnel 
involved. 

The  subject  of  personnel  motivation  could  be  closely 
associated  with  contract  incentives.  Using  the 
incentive  resources  to  reward  personal  performance, 
and  getting  it  passed  on  to  the  man  or  woman  on  the 
bench,  is  a very  important  aspect  of  mission 
assurance. 

Contractor/Government  Relations 

On  the  subject  of  Contractor/Government  relationship, 
the  discussion  centered  about  the  degree  of  Govern- 
ment engagement  vs  disengagement.  It  was  acknow- 
ledged that  the  DOD  climate  and  policy  in  this  area 
were  subject  to  change  from  time  to  time  and,  there- 
fore, establishing  an  appropriate  balance  between 
the  two  extremes  is  a matter  of  experience  and  sound 
judgment.  The  two  extremes  might  be  characterized 
by  a concept  similar  to  the  total  package  procure- 
ment on  one  hand,  and  on  the  other  hand  an  "engage- 
ment" activity  that  sees  the  Government  making 
contractors'  decisions  and  performing  contractor 
tasks. 

The  general  opinion  was  that  openness  on  both  sides 
was  a desirable  arrangement,  if  the  arrangement  is 
businesslike.  Conversely,  an  "adversary"  approach 
should  be  avoided. 

Many  factors  influenced  the  proper  engagement/ 
disengagement  balance,  such  as  the  stage  of  develop- 
ment, risks  involved,  type  of  contract,  state-of- 
the-art,  etc.  One  point  brought  up  during  these 
discussions  was  the  importance  of  attaining  complete 
understanding  between  the  Government  and  contractor 
personnel  concerning  all  aspects  of  newly  awarded 
contracts.  It  was  noted  that  often,  during  the 
source  selection  and  negotiating  phase,  personnel  on 
both  sides  are  not  the  same  personnel  as  those  who 
may  eventually  manage  the  contract.  The  need  for 
postaward  reviews  was  stressed  to  make  sure  both 
Government  and  Industry  start  with  a firm 
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understanding  of  the  contract  requirements  and 
tailored  application  of  specifications  and  stan- 
dards. This  postaward  review  should  include  those 
functional  management  and  contractual  areas  in 
which  misunderstandings  are  most  likely  to  occur. 

Type  of  Contract 

With  the  DOD  emphasis  on  fixed  price  contracts,  the 
discussion  centered  on  whether  or  not  the  type  of 
contract  for  fabrication  of  hardware  could  affect 
performance.  Primarily,  the  question  was:  Does  a 
fixed  price  environment  lead  the  contractor's  work 
force  to  make  decisions,  i.e.,  take  shortcuts  that 
inadvertently  contribute  to  performance  degradation, 
or  fail  to  surface  latent  defects  that  later  result 
in  failures  or  anomalies. 

As  might  be  expected,  the  opinion  was  divided  on 
this  question.  The  question  is  complex  and  many 
factors  are  involved,  such  as  the  stability  of 
design,  producibility  of  the  design,  schedule, 
incentive  structure,  and  validity  of  the  cost 
estimate.  It  was  felt  that  when  risks  are  high,  a 
cost-plus  arrangement  is  appropriate  but  steps  must 
be  taken  to  minimize  "buying  in." 

While  it  was  generally  agreed  that  it  was  not  wise 
for  industry  to  gamble  on  the  unknowns,  it  was 
noted  that  some  companies  have  had  favorable  exper- 
ience with  fixed  price  contracts  in  the  commercial 
satellite  business,  particularly  when  most  risks 
are  known. 

During  this  discussion,  a point  was  made  that  mis- 
sion success  and  successful  programs  often  tend  to 
lead  SAMSO  to  believe  that  costs  can  then  be 
reduced.  Some  participants  felt  that  attempting  to 
reduce  costs  on  successful  programs  may  cause 
degradation  to  the  successful  team  and  might  lead 
to  future  problems. 

Design 

One  of  the  factors  affecting  the  design  of  modern 
military  spacecraft  is  that  of  increasing  complex- 
ity. This  is  caused  in  part  by  the  increasing 
number  of  functions  a spacecraft  is  required  to 
perform.  Often  the  concept  of  operations  becomes 
overly  optimistic,  resulting  in  various  agencies 
adding  on  functions  and  capabilities.  The  result- 
ing increase  in  complexity  and  parts  count  has  a 
"snowball"  effect  on  the  entire  process  from  design 
through  launch.  While  there  is  no  single  most 
important  factor  to  consider  in  the  design  phase, 
it  was  felt  that  the  design  sets  the  stage  for 
mission  success.  The  following  are  highlights  of 
various  practices  and  ideas  suggested. 

• Achievement  of  mission  success  should  be 
made  a primary  design  goal.  During  the 
early  design  phase,  a complete  assessment 
of  all  risks  should  be  accomplished  to 
make  sure  the  approach  is  realist...  The 
general  feeling  was  that  overly  optimistic 
design  approaches  often  result  in  "build- 
ing in"  latent  defects. 

• ' '«•  point  was  made  that  the  desire  to 

i'  •'  the  maximum  capability  and  space- 

performance  could  result  in  "over 
- ‘f '.at  ion."  The  effort  to  achieve 
► (.•  rrcent  of  performance  may  often 

• < a"d  counterproductive. 


a The  Failure  Modes  and  Effects  Analysis 
(FMEA)  was  considered  an  important  tool 
for  structuring  the  design  process.  The 
FMEA  should  be  integrated  into  design 
trade-off  studies  to  make  sure  potential 
single  point  failures  are  controlled.  The 
FMEA  is  also  a good  basis  for  establishing 
inspection,  test,  and  critical  process 
requirements  for  monitoring  during  fabri- 
cation. Many  attendees  stressed  the 
importance  of  keeping  the  FMEA  updated 
whenever  any  design  changes  are  made. 

• The  use  of  redundancy  was  considered  to  be 
a good  approach  for  control  of  potential 
single  point  failures.  However,  it  was 
pointed  out  that  redundancy  should  not  be 
used  as  a crutch  for  weak  design  solutions. 

• Several  attendees  stated  that  requiring  an 
independent  assessment  of  the  spacecraft 
design  was  a useful  approach  to  assure 
that  as  many  potential  problems  as  possible 
are  eliminated  at  the  outset. 

• The  importance  of  not  leaving  design  prob- 
lems to  be  resolved  during  manufacture  was 
stressed.  It  was  noted  that  this  problem 
can  be  mitigated  by  requiring  manufacturing 
and  quality  assurance  "sign  off"  on  the 
design  prior  to  release  to  production. 

• There  was  a discussion  concerning  the 
necessity  or  desirability  of  including 
features  to  accommodate  repair,  replacement 
and  maintenance  in  spacecraft  design.  It 
has  been  generally  believed  in  the  past 
that  such  requirements  were  not  required 
for  space  applications.  However,  many 
instances  were  pointed  out  where  repair  or 
replacement  of  parts  were  required  before 
launch  and  such  features  would  have  been 
helpful.  It  was  also  noted  that  whenever 
long-term  storage  of  spacecraft  was  contem- 
plated, design  features  to  accommodate 
repair  should  be  considered. 

Testing 

Most  attendees  felt  that  thorough  and  extensive  test- 
ing of  parts,  components  and  subsystems  should  be 
planned  and  adequately  scheduled  from  the  outset. 
Adequate  testing  at  lower  levels  of  assembly,  at 
proper  environmental  parameters,  reduces  problems 
at  system  level  tests  and  avoids  overall  schedule 
impact  by  reducing  rework  and  retest. 

A strong  case  was  made  for  exercise  of  the  equipment 
in  the  same  manner  as  it  is  expected  to  be  operated 
on-orbit,  including  all  redundant  paths,  backup 
modes,  and  the  exercising  of  on-board  fault  detection 
circuitry.  Monitor  functions  continuously  so  that 
transients  that  might  be  indicative  of  latent  prob- 
lems are  not  missed.  Including  anomaly  simulation 
into  the  testing  techniques  is  also  considered  to 
have  merit.  It  is  especially  important  the  program 
schedule  provide  adequate  time  for  testing  at  all 
levels  of  assembly. 

It  was  considered  very  important  to  accomplish  a 
thorough  analysis  of  test  data  to  assure  potential 
failures  and  latent  defects  are  identified  and 
ferreted  out.  Experience  has  also  shown  that  iden- 
tifying and  replacing  marginal  parts  at  lower  levels 


of  assembly  decreases  costs  significantly  in 
downstream  activity.  In  the  same  vein,  the  root 
cause  of  all  failures  ar.d  anomalies  encountered  dur- 
ing testing  must  be  identified  and  corrected. 

Reviews  and  Audits, 

Many  contractors  have  established  separate  "mission 
success"  organizations  apart  from  the  classical 
functional  organizations.  Such  organizations  not 
only  perform  technical  audits,  they  serve  as  a mov- 
ing force  for  motivation  and  discipline  to  assure 
all  appropriate  actions  are  taken  and  mission  suc- 
cess visibility  is  retained  at  a higher  level. 
However,  it  was  pointed  out  that  caution  must  be 
exercised  to  avoid  committee  action  from  replacing 
individual  motivation  and  responsibility. 

Purchased  Electronic  Piece  Parts 

It  was  generally  agreed  that  obtaining  reliable 
piece  parts  is  a continuing  problem  for  the  space 
industry.  The  primary  cause  appears  to  be  the 
small  quantity  purchased  by  individual  contractors, 
and  therefore,  vendors  are  reluctant  to  initiate 
unprofitable  processes  and  controls. 

Stringent  measures  and  control  of  piece  part  vendors 
and  subcontractors  are  essential  to  obtain  reliable 
parts.  Increased  source  inspection  and  monitoring 
of  the  vendors'  processes  are  necessary  to  minimize 
problems.  Rigorous  testing,  analysis  of  samples, 
and  special  screening  inspections  are  essential. 
Monitored  production  lines  for  microelectronic  piece 
parts  indicates  that  significantly  improved  reli- 
ability can  be  achieved. 

The  importance  of  proper  screening  and  elimination 
of  marginal  parts  is  essential  to  preclude  the 
introduction  of  new  problems  when  faulty  parts  are 
"changed  out"  during  production. 

At  SAMSO,  the  Commander  has  established  a "Select 
Committee  on  Piece  Parts"  to  study  all  aspects  of 
this  problem.  The  committee  is  considering  several 
alternatives  and  changes  to  standard  ways  of  doing 
business  for  this  aspect  of  program  management. 

This  committee  has  and  will  continue  to  interface 
with  SAMSO  contractors  to  exchange  information  on 
piece  part  reliability. 

Manufacturing  and  Quality  Assurance 


to  identify  and  correct  root  causes,  is  essential 
for  mitigation  of  latent  defects. 

Some  attendees  indicated  that  quality  assurance  and 
configuration  control  of  software  is  becoming 
increasingly  important.  Because  of  the  increasing 
complexity  of  spacecraft,  testing  is  being  auto- 
mated at  a rapid  pace.  It  is  important  that  the 
nondeliverable  software  be  subjected  to  the  same 
controls  and  discipline  as  deliverable  software 
and  hardware. 

Experience  Shari ng/Communi cations 

Many  attendees  expressed  the  need  for  a more  effec- 
tive means  of  sharing  experience,  both  among  con- 
tractors and  between  the  contractors  and  SAMSO/ 
Aerospace.  Experience  sharing,  it  was  felt,  should 
include  both  success  information  as  well  as  adverse 
experiences  in  the  form  of  "lessons  learned." 
Industry  guests  expressed  their  appreciation  for  the 
opportunity  to  meet  with  the  SAMSO  Commander  and 
their  industry  counterparts.  Other  specific  sug- 
gestions included: 

• Holding  similar  meetings  periodically. 

c Having  broader  base  meetings  at  program 
manager  and  functional  speciality  level. 

• A better  means  of  informing  contractors  of 
the  SAMSO  experience  with  their  equipment 
should  be  explored. 

• Flight  failure  and  anomaly  information 
should  be  passed  on  to  all  contractors 
affected. 

t The  use  of  ALERTS  should  be  studied  with 
the  view  of  providing  more  useful  infor- 
mation to  contractors. 

• We  might  look  into  the  airline  industry's 
method  of  disseminating  problem 
information. 


It  was  felt  that  low  volume,  high  reliability 
spacecraft  and  booster  programs  present  unique 
problems.  Experienced  high-caliber  technicians 
must  be  used  for  fabrication  and  test,  and  often  it 
is  difficult  to  maintain  continuity  of  an  exper- 
ienced team.  The  transition  from  design  to  fabri- 
cation requires  more  attention  to  work  instructions 
to  assure  the  inherent  quality  is  not  degraded 
during  manufacture. 

Inspection  planning  and  inspectabil ity  should  be 
given  careful  consideration  during  design  and  plan- 
ning phases.  Relating  the  Failure  Mode  and  Effects 
Analysis  (FMEA)  to  the  methods  of  manufacturing, 
assembly.  Inspection  and  test  that  will  be  employed 
for  the  various  elements  of  a system  is  essential. 

Quality  assurance  personnel  need  to  accept  the 
challenge  of  extending  their  role  beyond  "conform- 
ance" to  that  of  Improving  the  design  and  processes. 
Also,  in-depth  analysis  of  defects  and  discrepancies, 
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